From: Alex Shi <lkml.alex@gmail.com>
To: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
David Rientjes <rientjes@google.com>,
Mel Gorman <mgorman@suse.de>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Paul Turner <pjt@google.com>,
Lee Schermerhorn <Lee.Schermerhorn@hp.com>,
Christoph Lameter <cl@linux.com>, Rik van Riel <riel@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
Johannes Weiner <hannes@cmpxchg.org>,
Hugh Dickins <hughd@google.com>, Alex Shi <alex.shi@intel.com>
Subject: Re: [PATCH, v2] mm, numa: Turn 4K pte NUMA faults into effective hugepage ones
Date: Fri, 23 Nov 2012 09:26:08 +0800 [thread overview]
Message-ID: <CAGjg+kHtdFE9Nc9ZTRjf73zwrOV77T=uX3ojsP=FWt8wbc2WBQ@mail.gmail.com> (raw)
In-Reply-To: <20121120160918.GA18167@gmail.com>
This patch cause boot hang on our SNB EP 2 sockets machine with some
segmentation fault.
revert it recovers booting.
============
[ 8.290147] Freeing unused kernel memory: 1264k freed
[ 8.306140] Freeing unused kernel memory: 1592k freed
[ 8.342668] init[250]: segfault at 20da510 ip 00000000020da510 sp
00007fff26788040 error 15[ 8.350983] usb 2-1: New USB device found,
idVendor=8087, idProduct=0024
[ 8.350987] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 8.351266] hub 2-1:1.0: USB hub found
[ 8.351346] hub 2-1:1.0: 8 ports detected
Segmentation fault
[ 8.626633] usb 2-1.4: new full-speed USB device number 3 using ehci_hcd
[ 8.721391] usb 2-1.4: New USB device found, idVendor=046b, idProduct=ff10
[ 8.729536] usb 2-1.4: New USB device strings: Mfr=1, Product=2,
SerialNumber=3
[ 8.738540] usb 2-1.4: Product: Virtual Keyboard and Mouse
[ 8.745134] usb 2-1.4: Manufacturer: American Megatrends Inc.
[ 8.752026] usb 2-1.4: SerialNumber: serial
[ 8.758877] input: American Megatrends Inc. Virtual Keyboard and
Mouse as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.0/input/input1
[ 8.774428] hid-generic 0003:046B:FF10.0001: input,hidraw0: USB HID
v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse]
on usb-0000:00:1d.0-1.4/input0
[ 8.793393] input: American Megatrends Inc. Virtual Keyboard and
Mouse as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.1/input/input2
[ 8.809140] hid-generic 0003:046B:FF10.0002: input,hidraw1: USB HID
v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on
usb-0000:00:1d.0-1.4/input1
[ 8.899511] usb 2-1.7: new low-speed USB device number 4 using ehci_hcd
[ 9.073473] usb 2-1.7: New USB device found, idVendor=0557, idProduct=2220
[ 9.081633] usb 2-1.7: New USB device strings: Mfr=1, Product=2,
SerialNumber=0
[ 9.090643] usb 2-1.7: Product: ATEN CS-1758/54
[ 9.096258] usb 2-1.7: Manufacturer: ATEN
[ 9.134093] input: ATEN ATEN CS-1758/54 as
/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.7/2-1.7:1.0/input/input3
[ 9.146804] hid-generic 0003:0557:2220.0003: input,hidraw2: USB HID
v1.10 Keyboard [ATEN ATEN CS-1758/54] on usb-0000:00:1d.0-1.7/input0
[ 9.184396] input: ATEN ATEN CS-1758/54 as
/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.7/2-1.7:1.1/input/input4
[ 9.197210] hid-generic 0003:0557:2220.0004: input,hidraw3: USB HID
v1.10 Mouse [ATEN ATEN CS-1758/54] on usb-0000:00:1d.0-1.7/input1
<hang here>
On Wed, Nov 21, 2012 at 12:09 AM, Ingo Molnar <mingo@kernel.org> wrote:
>
> Ok, the patch withstood a bit more testing as well. Below is a
> v2 version of it, with a couple of cleanups (no functional
> changes).
>
> Thanks,
>
> Ingo
>
> ----------------->
> Subject: mm, numa: Turn 4K pte NUMA faults into effective hugepage ones
> From: Ingo Molnar <mingo@kernel.org>
> Date: Tue Nov 20 15:48:26 CET 2012
>
> Reduce the 4K page fault count by looking around and processing
> nearby pages if possible.
>
> To keep the logic and cache overhead simple and straightforward
> we do a couple of simplifications:
>
> - we only scan in the HPAGE_SIZE range of the faulting address
> - we only go as far as the vma allows us
>
> Also simplify the do_numa_page() flow while at it and fix the
> previous double faulting we incurred due to not properly fixing
> up freshly migrated ptes.
>
> Suggested-by: Mel Gorman <mgorman@suse.de>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Cc: Andrea Arcangeli <aarcange@redhat.com>
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> ---
> mm/memory.c | 99 ++++++++++++++++++++++++++++++++++++++----------------------
> 1 file changed, 64 insertions(+), 35 deletions(-)
>
> Index: linux/mm/memory.c
> ===================================================================
> --- linux.orig/mm/memory.c
> +++ linux/mm/memory.c
> @@ -3455,64 +3455,93 @@ static int do_nonlinear_fault(struct mm_
> return __do_fault(mm, vma, address, pmd, pgoff, flags, orig_pte);
> }
>
> -static int do_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
> +static int __do_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
> unsigned long address, pte_t *ptep, pmd_t *pmd,
> - unsigned int flags, pte_t entry)
> + unsigned int flags, pte_t entry, spinlock_t *ptl)
> {
> - struct page *page = NULL;
> - int node, page_nid = -1;
> - int last_cpu = -1;
> - spinlock_t *ptl;
> -
> - ptl = pte_lockptr(mm, pmd);
> - spin_lock(ptl);
> - if (unlikely(!pte_same(*ptep, entry)))
> - goto out_unlock;
> + struct page *page;
> + int new_node;
>
> page = vm_normal_page(vma, address, entry);
> if (page) {
> - get_page(page);
> - page_nid = page_to_nid(page);
> - last_cpu = page_last_cpu(page);
> - node = mpol_misplaced(page, vma, address);
> - if (node != -1 && node != page_nid)
> + int page_nid = page_to_nid(page);
> + int last_cpu = page_last_cpu(page);
> +
> + task_numa_fault(page_nid, last_cpu, 1);
> +
> + new_node = mpol_misplaced(page, vma, address);
> + if (new_node != -1 && new_node != page_nid)
> goto migrate;
> }
>
> -out_pte_upgrade_unlock:
> +out_pte_upgrade:
> flush_cache_page(vma, address, pte_pfn(entry));
> -
> ptep_modify_prot_start(mm, address, ptep);
> entry = pte_modify(entry, vma->vm_page_prot);
> + if (pte_dirty(entry))
> + entry = pte_mkwrite(entry);
> ptep_modify_prot_commit(mm, address, ptep, entry);
> -
> /* No TLB flush needed because we upgraded the PTE */
> -
> update_mmu_cache(vma, address, ptep);
> -
> -out_unlock:
> - pte_unmap_unlock(ptep, ptl);
> -
> - if (page) {
> - task_numa_fault(page_nid, last_cpu, 1);
> - put_page(page);
> - }
> out:
> return 0;
>
> migrate:
> + get_page(page);
> pte_unmap_unlock(ptep, ptl);
>
> - if (migrate_misplaced_page(page, node)) {
> + migrate_misplaced_page(page, new_node); /* Drops the page reference */
> +
> + /* Re-check after migration: */
> +
> + ptl = pte_lockptr(mm, pmd);
> + spin_lock(ptl);
> + entry = ACCESS_ONCE(*ptep);
> +
> + if (!pte_numa(vma, entry))
> goto out;
> - }
> - page = NULL;
>
> - ptep = pte_offset_map_lock(mm, pmd, address, &ptl);
> - if (!pte_same(*ptep, entry))
> - goto out_unlock;
> + goto out_pte_upgrade;
> +}
> +
> +/*
> + * Add a simple loop to also fetch ptes within the same pmd:
> + */
> +static int do_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
> + unsigned long addr0, pte_t *ptep0, pmd_t *pmd,
> + unsigned int flags, pte_t entry0)
> +{
> + unsigned long addr0_pmd;
> + unsigned long addr_start;
> + unsigned long addr;
> + spinlock_t *ptl;
> + pte_t *ptep;
> +
> + addr0_pmd = addr0 & PMD_MASK;
> + addr_start = max(addr0_pmd, vma->vm_start);
>
> - goto out_pte_upgrade_unlock;
> + ptep = pte_offset_map(pmd, addr_start);
> + ptl = pte_lockptr(mm, pmd);
> + spin_lock(ptl);
> +
> + for (addr = addr_start; addr < vma->vm_end; addr += PAGE_SIZE, ptep++) {
> + pte_t entry;
> +
> + entry = ACCESS_ONCE(*ptep);
> +
> + if ((addr & PMD_MASK) != addr0_pmd)
> + break;
> + if (!pte_present(entry))
> + continue;
> + if (!pte_numa(vma, entry))
> + continue;
> +
> + __do_numa_page(mm, vma, addr, ptep, pmd, flags, entry, ptl);
> + }
> +
> + pte_unmap_unlock(ptep, ptl);
> +
> + return 0;
> }
>
> /*
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
--
Thanks
Alex
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2012-11-23 1:26 UTC|newest]
Thread overview: 101+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-19 2:14 [PATCH 00/27] Latest numa/core release, v16 Ingo Molnar
2012-11-19 2:14 ` [PATCH 01/27] mm/generic: Only flush the local TLB in ptep_set_access_flags() Ingo Molnar
2012-11-19 2:14 ` [PATCH 02/27] x86/mm: Only do a local tlb flush " Ingo Molnar
2012-11-19 2:14 ` [PATCH 03/27] x86/mm: Introduce pte_accessible() Ingo Molnar
2012-11-19 2:14 ` [PATCH 04/27] mm: Only flush the TLB when clearing an accessible pte Ingo Molnar
2012-11-19 2:14 ` [PATCH 05/27] x86/mm: Completely drop the TLB flush from ptep_set_access_flags() Ingo Molnar
2012-11-19 2:14 ` [PATCH 06/27] mm: Count the number of pages affected in change_protection() Ingo Molnar
2012-11-19 2:14 ` [PATCH 07/27] mm: Optimize the TLB flush of sys_mprotect() and change_protection() users Ingo Molnar
2012-11-19 2:14 ` [PATCH 08/27] sched, numa, mm: Add last_cpu to page flags Ingo Molnar
2012-11-19 2:14 ` [PATCH 09/27] sched, mm, numa: Create generic NUMA fault infrastructure, with architectures overrides Ingo Molnar
2012-11-19 2:14 ` [PATCH 10/27] sched: Make find_busiest_queue() a method Ingo Molnar
2012-11-19 2:14 ` [PATCH 11/27] sched, numa, mm: Describe the NUMA scheduling problem formally Ingo Molnar
2012-11-19 2:14 ` [PATCH 12/27] numa, mm: Support NUMA hinting page faults from gup/gup_fast Ingo Molnar
2012-11-19 2:14 ` [PATCH 13/27] mm/migrate: Introduce migrate_misplaced_page() Ingo Molnar
2012-11-19 2:14 ` [PATCH 14/27] sched, numa, mm, arch: Add variable locality exception Ingo Molnar
2012-11-19 2:14 ` [PATCH 15/27] sched, numa, mm: Add credits for NUMA placement Ingo Molnar
2012-11-19 2:14 ` [PATCH 16/27] sched, mm, x86: Add the ARCH_SUPPORTS_NUMA_BALANCING flag Ingo Molnar
2012-11-19 2:14 ` [PATCH 17/27] sched, numa, mm: Add the scanning page fault machinery Ingo Molnar
2012-11-19 2:14 ` [PATCH 18/27] sched: Add adaptive NUMA affinity support Ingo Molnar
2012-11-19 2:14 ` [PATCH 19/27] sched: Implement constant, per task Working Set Sampling (WSS) rate Ingo Molnar
2012-11-19 2:14 ` [PATCH 20/27] sched, numa, mm: Count WS scanning against present PTEs, not virtual memory ranges Ingo Molnar
2012-11-19 2:14 ` [PATCH 21/27] sched: Implement slow start for working set sampling Ingo Molnar
2012-11-19 2:14 ` [PATCH 22/27] sched, numa, mm: Interleave shared tasks Ingo Molnar
2012-11-19 2:14 ` [PATCH 23/27] sched: Implement NUMA scanning backoff Ingo Molnar
2012-11-19 2:14 ` [PATCH 24/27] sched: Improve convergence Ingo Molnar
2012-11-19 2:14 ` [PATCH 25/27] sched: Introduce staged average NUMA faults Ingo Molnar
2012-11-19 2:14 ` [PATCH 26/27] sched: Track groups of shared tasks Ingo Molnar
2012-11-19 2:14 ` [PATCH 27/27] sched: Use the best-buddy 'ideal cpu' in balancing decisions Ingo Molnar
2012-11-19 16:29 ` [PATCH 00/27] Latest numa/core release, v16 Mel Gorman
2012-11-19 19:13 ` Ingo Molnar
2012-11-19 21:18 ` Mel Gorman
2012-11-19 22:36 ` Ingo Molnar
2012-11-19 23:00 ` Mel Gorman
2012-11-20 0:41 ` Rik van Riel
2012-11-21 10:58 ` Mel Gorman
2012-11-20 1:02 ` Linus Torvalds
2012-11-20 7:17 ` Ingo Molnar
2012-11-20 7:37 ` David Rientjes
2012-11-20 7:48 ` Ingo Molnar
2012-11-20 8:01 ` Ingo Molnar
2012-11-20 8:11 ` David Rientjes
2012-11-21 11:14 ` Mel Gorman
2012-11-20 10:20 ` Mel Gorman
2012-11-20 10:47 ` Mel Gorman
2012-11-20 15:29 ` [PATCH] mm, numa: Turn 4K pte NUMA faults into effective hugepage ones Ingo Molnar
2012-11-20 16:09 ` [PATCH, v2] " Ingo Molnar
2012-11-20 16:31 ` Rik van Riel
2012-11-20 16:52 ` Ingo Molnar
2012-11-21 12:08 ` Mel Gorman
2012-11-21 8:12 ` Ingo Molnar
2012-11-21 2:41 ` David Rientjes
2012-11-21 9:34 ` Ingo Molnar
2012-11-21 11:40 ` Mel Gorman
2012-11-23 1:26 ` Alex Shi [this message]
2012-11-20 17:56 ` numa/core regressions fixed - more testers wanted Ingo Molnar
2012-11-21 1:54 ` Andrew Theurer
2012-11-21 3:22 ` Rik van Riel
2012-11-21 4:10 ` Hugh Dickins
2012-11-21 17:59 ` Andrew Theurer
2012-11-21 11:52 ` Mel Gorman
2012-11-21 22:15 ` Andrew Theurer
2012-11-21 3:33 ` David Rientjes
2012-11-21 9:38 ` Ingo Molnar
2012-11-21 11:06 ` Ingo Molnar
2012-11-21 8:39 ` Alex Shi
2012-11-22 1:21 ` Ingo Molnar
2012-11-23 13:31 ` Ingo Molnar
2012-11-23 15:23 ` Alex Shi
2012-11-26 2:11 ` Alex Shi
2012-11-28 14:21 ` Alex Shi
2012-11-20 10:40 ` [PATCH 00/27] Latest numa/core release, v16 Ingo Molnar
2012-11-20 11:40 ` Mel Gorman
2012-11-21 10:38 ` Mel Gorman
2012-11-21 19:37 ` Andrea Arcangeli
2012-11-21 19:56 ` Mel Gorman
2012-11-19 20:07 ` Ingo Molnar
2012-11-19 21:37 ` Mel Gorman
2012-11-20 0:50 ` David Rientjes
2012-11-20 1:05 ` David Rientjes
2012-11-20 6:00 ` Ingo Molnar
2012-11-20 6:20 ` David Rientjes
2012-11-20 7:44 ` Ingo Molnar
2012-11-20 7:48 ` Paul Turner
2012-11-20 8:20 ` David Rientjes
2012-11-20 9:06 ` Ingo Molnar
2012-11-20 9:41 ` [patch] x86/vsyscall: Add Kconfig option to use native vsyscalls, switch to it Ingo Molnar
2012-11-20 23:01 ` Andy Lutomirski
2012-11-21 0:43 ` David Rientjes
2012-11-20 12:02 ` [PATCH] x86/mm: Don't flush the TLB on #WP pmd fixups Ingo Molnar
2012-11-20 12:31 ` Ingo Molnar
2012-11-21 11:47 ` Mel Gorman
2012-11-21 1:22 ` David Rientjes
2012-11-21 17:02 ` [PATCH 00/27] Latest numa/core release, v16 Linus Torvalds
2012-11-21 17:10 ` Ingo Molnar
2012-11-21 17:20 ` Ingo Molnar
2012-11-22 4:31 ` David Rientjes
2012-11-21 17:40 ` Ingo Molnar
2012-11-21 22:04 ` Linus Torvalds
2012-11-21 22:46 ` Ingo Molnar
2012-11-21 17:45 ` Rik van Riel
2012-11-21 18:04 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGjg+kHtdFE9Nc9ZTRjf73zwrOV77T=uX3ojsP=FWt8wbc2WBQ@mail.gmail.com' \
--to=lkml.alex@gmail.com \
--cc=Lee.Schermerhorn@hp.com \
--cc=a.p.zijlstra@chello.nl \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=alex.shi@intel.com \
--cc=cl@linux.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=pjt@google.com \
--cc=riel@redhat.com \
--cc=rientjes@google.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox