From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02A1DC6379F for ; Tue, 21 Feb 2023 12:52:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 676096B0073; Tue, 21 Feb 2023 07:52:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5FF196B0074; Tue, 21 Feb 2023 07:52:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4793F6B0075; Tue, 21 Feb 2023 07:52:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 342B96B0073 for ; Tue, 21 Feb 2023 07:52:39 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8B0AF8016C for ; Tue, 21 Feb 2023 12:52:38 +0000 (UTC) X-FDA: 80491287996.03.B7B54E8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf21.hostedemail.com (Postfix) with ESMTP id AD2F11C001E for ; Tue, 21 Feb 2023 12:52:36 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hgCO0M8J; spf=pass (imf21.hostedemail.com: domain of bhe@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bhe@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676983956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cim37CPurlllaqqM13IPZavfyIJo0MFG0mwjskCxTms=; b=IxEC0yhFtr/FFK+kVB23/w0k+M2UT8Ouc2XmIk2rErIsus1ZTUQyo6bBJfbK82thkAY5MK 98xTSXlgE3Hc4TWlZVe4M6VdUDoIVzu6YFCnok6OXfalOL1Jg1noEamFOjw8+TDidORYKB cxfTf13Es8KgI7ktaHbX5T024//rUN0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hgCO0M8J; spf=pass (imf21.hostedemail.com: domain of bhe@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bhe@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676983956; a=rsa-sha256; cv=none; b=nusvidyK4hDKxTauhCLRpuuxVnxAJKzPTowtuiZYu167iWxYisV4QMPlZz7mrCpLhK6jYQ uSyTpLaUcj6X9aUNxuhVIQUKZIK+jPC7PcYGGdIijNYmInBwJ77m7DKTJxufwwHkq8fvdN ylmA4pXb5pKj/gmKXBy8Ug4mNp2qHB8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676983956; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cim37CPurlllaqqM13IPZavfyIJo0MFG0mwjskCxTms=; b=hgCO0M8Jn6+MsQZXTM1w/D1P+zlXodCovHISeKd/O71wi30pu0oN8COxD8CPkKvbM1h+5j JRzGlDCAXlOTkrDTNfDjMou1FB49MYQLSZweCkSP1cc+qdQlOY/B0GFY6edLHn+U4khpbX aRSM7zbZ3szegbWrtGmWdlfm5+wyj3U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-327-SiXvUBB0N5SiMGhbDklnUA-1; Tue, 21 Feb 2023 07:52:32 -0500 X-MC-Unique: SiXvUBB0N5SiMGhbDklnUA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C5FE385A5A3; Tue, 21 Feb 2023 12:52:31 +0000 (UTC) Received: from localhost (ovpn-13-7.pek2.redhat.com [10.72.13.7]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F08472166B26; Tue, 21 Feb 2023 12:52:30 +0000 (UTC) Date: Tue, 21 Feb 2023 20:52:27 +0800 From: Baoquan He To: Niklas Schnelle Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, christophe.leroy@csgroup.eu, hch@infradead.org, agordeev@linux.ibm.com, wangkefeng.wang@huawei.com, David.Laight@aculab.com, shorne@gmail.com, arnd@arndb.de, Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , linux-s390@vger.kernel.org Subject: Re: [PATCH v4 09/16] s390: mm: Convert to GENERIC_IOREMAP Message-ID: References: <20230216123419.461016-1-bhe@redhat.com> <20230216123419.461016-10-bhe@redhat.com> <9bb1154b07dc21e5d3dda8cc5238c5385f32c2e0.camel@linux.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Stat-Signature: ct1nznr6zqh358sojbdcpok5pb8r8bhk X-Rspam-User: X-Rspamd-Queue-Id: AD2F11C001E X-Rspamd-Server: rspam06 X-HE-Tag: 1676983956-783576 X-HE-Meta: U2FsdGVkX1/CQw4wZ5kvNxPiXH1Rf5/nJgU5EioehfXI3CTq6mZ0UKHKZK+9wpxVfw3CmK+vOEs31uKhcHzdi0lyRhaAZWS0myBEaq/KSm0gZ4Al7sXNnFDI7MYyNQOlBjWDyaVfFDOPFG7gNU8ybkd7uA1S/wnhtVZyxMlO1Bj8qu8iMIvQHq3t2jdYAhbI3PzbyPlcKHMmlW+W6DbX2g6b0pnJyUVGGsqBgb6e8ox8JL3sec+tKg78+AJ9V/ZFbxphPl4u8TOYqLhD+XXTefDQC98g3y38Vyz2ouRbtoN5U2l4waFi+253odlgsBGQhgDG8GijXwXs4kiyIGA9iecLDL7tjFwRdusCysQ0VHMA1XuwRZRm0nHDGN2lQNUT0hHpeLFyMa12CVLYCDLK5CaQNo+/9pmbaKcKkoJSoq2Mzv3FIQXZg9/i71nR4Y8wQD/mkDiEdJuFQemvrWt44eabwyPEpzfu4HK8KvXGbr1gcK36373TDk7nTOoR3QHJEQR3VP2Gx74CWnKLrAFSYnxYbsLnucnVLqBD2b+kGs+1VZeglUsNQ9v6WJl9OHYXFH9dj8zxeVy9zkqsL+qutgMvQpNnswMxcIvjuqWQQgE0xqlgflaj9BmQBXTOSOHtJykUbuwYd683olI0jUgv5bjnTBEeCEWB341alGACZ8Zt2LmgKSxzWsjmEyah0bczeKU6z4bn48PKM4HgFE6dmpgsZsi3e3xlU7HfKGCg693rBxfbZadOt8gjvOamvc7ZRda5+0tluHhzHmmAomeuuuf86bfJUq2bSOtgGR8W5I5WqrLV3K4VuWFxSfT+kbWVNo+JdaONmuFo89Vql/T/aez1eXZpk9K07+96Rt8fJt1onv+2pGC/SGtvzPoVNcavsgpkB9UHA0hrovEgYGZDZtXxooCBiFdmmtdj+xwFgNtDDquLyk5ce9riXtPXcQBmXGEwFyILuKdU2NB7cvr o8FnDZa9 I12lOjVq+c8712UasfVUQytJhqrbG/FMtQTi5n2PSbtdi0B4ogsR03ZfieB2nZaAtIetb0k2Aq+8qL9RdcY88uZzOskvJG4gYweuuJgK7JBLDG3cKNhOyqPnIRLWLrZPCMiKvit9rmxIqyw8pcnFkBejPDmsucejLvaNflv0Qpu1Ktcp/6B4Pij6nGAiNV1SLmY5IZgrQTgUFxnHcSrZGlBEX3ly91f1nYFrGRCBlmrsPiTHSSuhS3C1XiKIxee9YGvIw1aIDKZk9+JN30BaSAjdd0ck7llqqvZzg60LDkU3bKlXrntB0N3G6bGyC0OEKjeScYQ09r4WQn/D8gAktZDZA3xtcJWbZ7BZoasd6K6MYQY905Oy1nb+P8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 02/21/23 at 01:26pm, Niklas Schnelle wrote: > On Tue, 2023-02-21 at 19:48 +0800, Baoquan He wrote: > > On 02/16/23 at 05:21pm, Niklas Schnelle wrote: > > > On Thu, 2023-02-16 at 20:34 +0800, Baoquan He wrote: > > > > By taking GENERIC_IOREMAP method, the generic generic_ioremap_prot(), > > > > generic_iounmap(), and their generic wrapper ioremap_prot(), ioremap() > > > > and iounmap() are all visible and available to arch. Arch needs to > > > > provide wrapper functions to override the generic versions if there's > > > > arch specific handling in its ioremap_prot(), ioremap() or iounmap(). > > > > This change will simplify implementation by removing duplicated codes > > > > with generic_ioremap_prot() and generic_iounmap(), and has the equivalent > > > > functioality as before. > > > > > > > > Here, add wrapper functions ioremap_prot() and iounmap() for s390's > > > > special operation when ioremap() and iounmap(). > > > > > > > > Signed-off-by: Baoquan He > > > > Cc: Niklas Schnelle > > > > Cc: Gerald Schaefer > > > > Cc: Heiko Carstens > > > > Cc: Vasily Gorbik > > > > Cc: Alexander Gordeev > > > > Cc: Christian Borntraeger > > > > Cc: Sven Schnelle > > > > Cc: linux-s390@vger.kernel.org > > > > --- > > > > arch/s390/Kconfig | 1 + > > > > arch/s390/include/asm/io.h | 21 ++++++++------ > > > > arch/s390/pci/pci.c | 57 +++++++------------------------------- > > > > 3 files changed, 23 insertions(+), 56 deletions(-) > > > > > > Thanks for the patch. This is a very clear improvement for us! I tested > > > this series with s390 systems with and without the PCI memory-I/O (MIO) > > > support and everything works as expected. > > > > > > One thing I did stumble upon but which is independent from this patch > > > is that I think memremap(…, MEMREMAP_WB) on system RAM outside the > > > direct map is broken for us. At least on systems without PCI memory-I/O > > > support. I think with this series it would be much easier to fix > > > though. Basically I think we would have to define arch_memremap_wb() to > > > generic_ioremap_prot(…, PAGE_KERNEL) and then have iounmap() check > > > is_ioremap_addr() to see if it is an actual mapping or an address > > > cookie. But again this is independent of this patch which doesn't > > > change the behavior in this area. > > > > OK, I can check this after this patchset done. > > Oh this wasn't meant as adding a task for you just a FYI. I'm already > experimenting with this and plan to possibly send a patch that fixes > the case of remapping system RAM once your series has landed. The thing > is that memremap() is quite tricky for us whichever way you look at > it.  > > We don't have real MMIO on s390x and thus using memremap() for PCI MMIO > spaces doesn't really work as the void* returned can still only be > accessed using ioread()/iowrite() because only our special PCI access > instructions work on the pseudo-MMIO addresses (with PCI MIO) > respectively the address cookies. Obviously void* and memremap() really > isn't the right tool if you need to use ioread()/iowrite() to access it > and can't treat it as memory though. Ah, I misunderstood it. Thanks for sharing and expect to see your post.