Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Alex Xu (Hello71)]

Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad 
rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to 
start filling dmesg, and then closing programs causes more BUGs and 
hangs, and then everything grinds to a halt (can't start more programs, 
can't even reboot through systemd).

Using master and reverting that branch up to that point fixes the 
problem.

I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4 
board with IOMMU enabled.

Re: Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Thomas Hellström (VMware)]

On 4/6/20 9:51 PM, Alex Xu (Hello71) wrote:
> Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad
> rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to
> start filling dmesg, and then closing programs causes more BUGs and
> hangs, and then everything grinds to a halt (can't start more programs,
> can't even reboot through systemd).
>
> Using master and reverting that branch up to that point fixes the
> problem.
>
> I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4
> board with IOMMU enabled.

Hmm. That sounds bad. Could you send a copy of your config?

Meanwhile, I'll prepare a small patch that disables the non-vmwgfx 
huge_fault() until we've figured out what's happening.

/Thomas

Re: Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Thomas Hellström (VMware)]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

Hi,

On 4/6/20 9:51 PM, Alex Xu (Hello71) wrote:
> Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad
> rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to
> start filling dmesg, and then closing programs causes more BUGs and
> hangs, and then everything grinds to a halt (can't start more programs,
> can't even reboot through systemd).
>
> Using master and reverting that branch up to that point fixes the
> problem.
>
> I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4
> board with IOMMU enabled.

If you could try the attached patch, that'd be great!

Thanks,

Thomas



[-- Attachment #2: 0001-drm-ttm-Temporarily-disable-the-huge_fault-callback.patch --]
[-- Type: text/x-patch, Size: 2774 bytes --]

From b630b9b4dcc1d01514d97a84cbb7f0cb85333154 Mon Sep 17 00:00:00 2001
From: "Thomas Hellstrom (VMware)" <thomas_os@shipmail.org>
Date: Mon, 6 Apr 2020 22:55:13 +0200
Subject: [PATCH] drm/ttm: Temporarily disable the huge_fault() callback

Signed-off-by: Thomas Hellstrom (VMware) <thomas_os@shipmail.org>
---
 drivers/gpu/drm/ttm/ttm_bo_vm.c | 63 ---------------------------------
 1 file changed, 63 deletions(-)

diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 6ee3b96f0d13..0ad30b112982 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -442,66 +442,6 @@ vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf)
 }
 EXPORT_SYMBOL(ttm_bo_vm_fault);
 
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-/**
- * ttm_pgprot_is_wrprotecting - Is a page protection value write-protecting?
- * @prot: The page protection value
- *
- * Return: true if @prot is write-protecting. false otherwise.
- */
-static bool ttm_pgprot_is_wrprotecting(pgprot_t prot)
-{
-	/*
-	 * This is meant to say "pgprot_wrprotect(prot) == prot" in a generic
-	 * way. Unfortunately there is no generic pgprot_wrprotect.
-	 */
-	return pte_val(pte_wrprotect(__pte(pgprot_val(prot)))) ==
-		pgprot_val(prot);
-}
-
-static vm_fault_t ttm_bo_vm_huge_fault(struct vm_fault *vmf,
-				       enum page_entry_size pe_size)
-{
-	struct vm_area_struct *vma = vmf->vma;
-	pgprot_t prot;
-	struct ttm_buffer_object *bo = vma->vm_private_data;
-	vm_fault_t ret;
-	pgoff_t fault_page_size = 0;
-	bool write = vmf->flags & FAULT_FLAG_WRITE;
-
-	switch (pe_size) {
-	case PE_SIZE_PMD:
-		fault_page_size = HPAGE_PMD_SIZE >> PAGE_SHIFT;
-		break;
-#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
-	case PE_SIZE_PUD:
-		fault_page_size = HPAGE_PUD_SIZE >> PAGE_SHIFT;
-		break;
-#endif
-	default:
-		WARN_ON_ONCE(1);
-		return VM_FAULT_FALLBACK;
-	}
-
-	/* Fallback on write dirty-tracking or COW */
-	if (write && ttm_pgprot_is_wrprotecting(vma->vm_page_prot))
-		return VM_FAULT_FALLBACK;
-
-	ret = ttm_bo_vm_reserve(bo, vmf);
-	if (ret)
-		return ret;
-
-	prot = vm_get_page_prot(vma->vm_flags);
-	ret = ttm_bo_vm_fault_reserved(vmf, prot, 1, fault_page_size);
-	if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
-		return ret;
-
-	dma_resv_unlock(bo->base.resv);
-
-	return ret;
-}
-#endif
-
 void ttm_bo_vm_open(struct vm_area_struct *vma)
 {
 	struct ttm_buffer_object *bo = vma->vm_private_data;
@@ -604,9 +544,6 @@ static const struct vm_operations_struct ttm_bo_vm_ops = {
 	.open = ttm_bo_vm_open,
 	.close = ttm_bo_vm_close,
 	.access = ttm_bo_vm_access,
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
-	.huge_fault = ttm_bo_vm_huge_fault,
-#endif
 };
 
 static struct ttm_buffer_object *ttm_bo_vm_lookup(struct ttm_bo_device *bdev,
-- 
2.21.1

Re: Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Alex Xu (Hello71)]

Excerpts from Thomas Hellström (VMware)'s message of April 6, 2020 5:04 pm:
> Hi,
> 
> On 4/6/20 9:51 PM, Alex Xu (Hello71) wrote:
>> Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad
>> rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to
>> start filling dmesg, and then closing programs causes more BUGs and
>> hangs, and then everything grinds to a halt (can't start more programs,
>> can't even reboot through systemd).
>>
>> Using master and reverting that branch up to that point fixes the
>> problem.
>>
>> I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4
>> board with IOMMU enabled.
> 
> If you could try the attached patch, that'd be great!
> 
> Thanks,
> 
> Thomas
> 

Yeah, that works too. Kernel config sent off-list.

Regards,
Alex.

Re: Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Thomas Hellström (VMware)]

On 4/7/20 2:38 AM, Alex Xu (Hello71) wrote:
> Excerpts from Thomas Hellström (VMware)'s message of April 6, 2020 5:04 pm:
>> Hi,
>>
>> On 4/6/20 9:51 PM, Alex Xu (Hello71) wrote:
>>> Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad
>>> rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to
>>> start filling dmesg, and then closing programs causes more BUGs and
>>> hangs, and then everything grinds to a halt (can't start more programs,
>>> can't even reboot through systemd).
>>>
>>> Using master and reverting that branch up to that point fixes the
>>> problem.
>>>
>>> I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4
>>> board with IOMMU enabled.
>> If you could try the attached patch, that'd be great!
>>
>> Thanks,
>>
>> Thomas
>>
> Yeah, that works too. Kernel config sent off-list.
>
> Regards,
> Alex.

Thanks. Do you want me to add your

Reported-by: and Tested-by: To this patch?

/Thomas

Re: Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Alex Xu (Hello71)]

Excerpts from Thomas Hellström (VMware)'s message of April 7, 2020 7:26 am:
> On 4/7/20 2:38 AM, Alex Xu (Hello71) wrote:
>> Excerpts from Thomas Hellström (VMware)'s message of April 6, 2020 5:04 pm:
>>> Hi,
>>>
>>> On 4/6/20 9:51 PM, Alex Xu (Hello71) wrote:
>>>> Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad
>>>> rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to
>>>> start filling dmesg, and then closing programs causes more BUGs and
>>>> hangs, and then everything grinds to a halt (can't start more programs,
>>>> can't even reboot through systemd).
>>>>
>>>> Using master and reverting that branch up to that point fixes the
>>>> problem.
>>>>
>>>> I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4
>>>> board with IOMMU enabled.
>>> If you could try the attached patch, that'd be great!
>>>
>>> Thanks,
>>>
>>> Thomas
>>>
>> Yeah, that works too. Kernel config sent off-list.
>>
>> Regards,
>> Alex.
> 
> Thanks. Do you want me to add your
> 
> Reported-by: and Tested-by: To this patch?
> 
> /Thomas
> 
> 

Sure. Shouldn't we fix it properly though?

Re: Bad rss-counter state from drm/ttm, drm/vmwgfx: Support huge TTM pagefaults

[Thomas Hellström (VMware)]

On 4/7/20 5:36 PM, Alex Xu (Hello71) wrote:
> Excerpts from Thomas Hellström (VMware)'s message of April 7, 2020 7:26 am:
>> On 4/7/20 2:38 AM, Alex Xu (Hello71) wrote:
>>> Excerpts from Thomas Hellström (VMware)'s message of April 6, 2020 5:04 pm:
>>>> Hi,
>>>>
>>>> On 4/6/20 9:51 PM, Alex Xu (Hello71) wrote:
>>>>> Using 314b658 with amdgpu, starting sway and firefox causes "BUG: Bad
>>>>> rss-counter state" and "BUG: non-zero pgtables_bytes on freeing mm" to
>>>>> start filling dmesg, and then closing programs causes more BUGs and
>>>>> hangs, and then everything grinds to a halt (can't start more programs,
>>>>> can't even reboot through systemd).
>>>>>
>>>>> Using master and reverting that branch up to that point fixes the
>>>>> problem.
>>>>>
>>>>> I'm using a Ryzen 1600 and AMD Radeon RX 480 on an ASRock B450 Pro4
>>>>> board with IOMMU enabled.
>>>> If you could try the attached patch, that'd be great!
>>>>
>>>> Thanks,
>>>>
>>>> Thomas
>>>>
>>> Yeah, that works too. Kernel config sent off-list.
>>>
>>> Regards,
>>> Alex.
>> Thanks. Do you want me to add your
>>
>> Reported-by: and Tested-by: To this patch?
>>
>> /Thomas
>>
>>
> Sure. Shouldn't we fix it properly though?

It's still enabled for vmwgfx for which it is reasonably well tested and 
where I can't see any such errors.

The code we remove with this patch enables huge page-table entries in 
some circumstances for other drivers, but given the problems you're 
seeing for amdgpu, it's better to enable this on a per-driver basis 
after thorough testing. Since I don't have amdgpu hardware I'm not sure 
what it's doing differently, and can't debug the issue properly.

/Thomas