From: "Larry H." <research@subreption.com>
To: Arjan van de Ven <arjan@infradead.org>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>, Ingo Molnar <mingo@elte.hu>,
Rik van Riel <riel@redhat.com>,
linux-kernel@vger.kernel.org, Linus Torvalds <torvalds@osdl.org>,
linux-mm@kvack.org, Ingo Molnar <mingo@redhat.com>,
pageexec@freemail.hu
Subject: [PATCH] Support for unconditional page sanitization
Date: Sat, 23 May 2009 11:21:41 -0700 [thread overview]
Message-ID: <20090523182141.GK13971@oblivion.subreption.com> (raw)
In-Reply-To: <20090523085653.0ad217f8@infradead.org>
[PATCH] Support for unconditional page sanitization
A boot command line option (sanitize_mem) is added for the page
allocator to perform sanitization of all pages upon release.
This avoids leaking sensitive information when memory is released to
the system after use, for example in cryptographic subsystems. More
specifically, the following threats are addressed:
1. Information leaks in use-after-free or uninitialized
variable usage scenarios, such as CVE-2005-0400,
CVE-2009-0787 and CVE-2007-6417.
2. Data remanence based attacks, such as Iceman/Coldboot,
which combine cold rebooting and memory image scanning
to extract cryptographic secrets (ex. detecting AES key
expansion blocks, RSA key patterns, etc) or other
confidential information.
3. Re-allocation based information leaks, especially in the
SLAB/SLUB allocators which use LIFO caches and might expose
sensitive data out of context (when a caller allocates an
object and receives a pointer to a location which was used
previously by another user).
The "Shredding Your Garbage: Reducing Data Lifetime Through Secure
Deallocation" paper by Jim Chow et. al from the Stanford University
Department of Computer Science, explains the security implications of
insecure deallocation, and provides extensive information with figures
and applications thoroughly analyzed for this behavior [1]. More recently
this issue came to widespread attention when the "Lest We Remember:
Cold Boot Attacks on Encryption Keys" (by Halderman et. al) paper was
published [2].
The code is largely based off the memory sanitization feature in the
PaX project (licensed under the GPL v2 terms) and the original
PG_sensitive patch which allowed fine-grained marking of pages using
a page flag.
This patch has been tested on x86 and amd64, with and without HIGHMEM.
[1] http://www.stanford.edu/~blp/papers/shredding.html
[2] http://citp.princeton.edu/memory/
[3] http://marc.info/?l=linux-mm&m=124284428226461&w=2
[4] http://marc.info/?t=124284431000002&r=1&w=2
Signed-off-by: Larry Highsmith <research@subreption.com>
---
Documentation/kernel-parameters.txt | 2 ++
arch/alpha/include/asm/kmap_types.h | 3 ++-
arch/arm/include/asm/kmap_types.h | 1 +
arch/avr32/include/asm/kmap_types.h | 3 ++-
arch/blackfin/include/asm/kmap_types.h | 1 +
arch/cris/include/asm/kmap_types.h | 1 +
arch/h8300/include/asm/kmap_types.h | 1 +
arch/ia64/include/asm/kmap_types.h | 3 ++-
arch/m68k/include/asm/kmap_types_mm.h | 1 +
arch/m68k/include/asm/kmap_types_no.h | 1 +
arch/mips/include/asm/kmap_types.h | 3 ++-
arch/parisc/include/asm/kmap_types.h | 3 ++-
arch/powerpc/include/asm/kmap_types.h | 1 +
arch/s390/include/asm/kmap_types.h | 1 +
arch/sh/include/asm/kmap_types.h | 3 ++-
arch/sparc/include/asm/kmap_types.h | 1 +
arch/um/include/asm/kmap_types.h | 1 +
arch/x86/include/asm/kmap_types.h | 3 ++-
arch/xtensa/include/asm/kmap_types.h | 1 +
include/asm-frv/kmap_types.h | 1 +
include/asm-m32r/kmap_types.h | 3 ++-
include/asm-mn10300/kmap_types.h | 1 +
include/linux/highmem.h | 12 ++++++++++++
mm/page_alloc.c | 27 ++++++++++++++++++++++++++-
24 files changed, 69 insertions(+), 9 deletions(-)
Index: linux-2.6/Documentation/kernel-parameters.txt
===================================================================
--- linux-2.6.orig/Documentation/kernel-parameters.txt
+++ linux-2.6/Documentation/kernel-parameters.txt
@@ -2494,6 +2494,8 @@ and is between 256 and 4096 characters.
norandmaps Don't use address space randomization. Equivalent to
echo 0 > /proc/sys/kernel/randomize_va_space
+ sanitize_mem Enables sanitization of all allocated pages.
+
______________________________________________________________________
TODO:
Index: linux-2.6/arch/alpha/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/alpha/include/asm/kmap_types.h
+++ linux-2.6/arch/alpha/include/asm/kmap_types.h
@@ -24,7 +24,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/arm/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/arm/include/asm/kmap_types.h
+++ linux-2.6/arch/arm/include/asm/kmap_types.h
@@ -18,6 +18,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/avr32/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/avr32/include/asm/kmap_types.h
+++ linux-2.6/arch/avr32/include/asm/kmap_types.h
@@ -22,7 +22,8 @@ D(10) KM_IRQ0,
D(11) KM_IRQ1,
D(12) KM_SOFTIRQ0,
D(13) KM_SOFTIRQ1,
-D(14) KM_TYPE_NR
+D(14) KM_CLEARPAGE,
+D(15) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/blackfin/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/blackfin/include/asm/kmap_types.h
+++ linux-2.6/arch/blackfin/include/asm/kmap_types.h
@@ -15,6 +15,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/cris/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/cris/include/asm/kmap_types.h
+++ linux-2.6/arch/cris/include/asm/kmap_types.h
@@ -19,6 +19,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/h8300/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/h8300/include/asm/kmap_types.h
+++ linux-2.6/arch/h8300/include/asm/kmap_types.h
@@ -15,6 +15,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/ia64/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/ia64/include/asm/kmap_types.h
+++ linux-2.6/arch/ia64/include/asm/kmap_types.h
@@ -22,7 +22,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/m68k/include/asm/kmap_types_mm.h
===================================================================
--- linux-2.6.orig/arch/m68k/include/asm/kmap_types_mm.h
+++ linux-2.6/arch/m68k/include/asm/kmap_types_mm.h
@@ -15,6 +15,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/m68k/include/asm/kmap_types_no.h
===================================================================
--- linux-2.6.orig/arch/m68k/include/asm/kmap_types_no.h
+++ linux-2.6/arch/m68k/include/asm/kmap_types_no.h
@@ -15,6 +15,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/mips/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/mips/include/asm/kmap_types.h
+++ linux-2.6/arch/mips/include/asm/kmap_types.h
@@ -22,7 +22,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/parisc/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/parisc/include/asm/kmap_types.h
+++ linux-2.6/arch/parisc/include/asm/kmap_types.h
@@ -22,7 +22,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/powerpc/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/powerpc/include/asm/kmap_types.h
+++ linux-2.6/arch/powerpc/include/asm/kmap_types.h
@@ -26,6 +26,7 @@ enum km_type {
KM_SOFTIRQ1,
KM_PPC_SYNC_PAGE,
KM_PPC_SYNC_ICACHE,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/s390/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/s390/include/asm/kmap_types.h
+++ linux-2.6/arch/s390/include/asm/kmap_types.h
@@ -16,6 +16,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/sh/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/sh/include/asm/kmap_types.h
+++ linux-2.6/arch/sh/include/asm/kmap_types.h
@@ -24,7 +24,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/sparc/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/sparc/include/asm/kmap_types.h
+++ linux-2.6/arch/sparc/include/asm/kmap_types.h
@@ -19,6 +19,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/um/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/um/include/asm/kmap_types.h
+++ linux-2.6/arch/um/include/asm/kmap_types.h
@@ -23,6 +23,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/arch/x86/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/kmap_types.h
+++ linux-2.6/arch/x86/include/asm/kmap_types.h
@@ -21,7 +21,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/arch/xtensa/include/asm/kmap_types.h
===================================================================
--- linux-2.6.orig/arch/xtensa/include/asm/kmap_types.h
+++ linux-2.6/arch/xtensa/include/asm/kmap_types.h
@@ -25,6 +25,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/include/asm-frv/kmap_types.h
===================================================================
--- linux-2.6.orig/include/asm-frv/kmap_types.h
+++ linux-2.6/include/asm-frv/kmap_types.h
@@ -23,6 +23,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/include/asm-m32r/kmap_types.h
===================================================================
--- linux-2.6.orig/include/asm-m32r/kmap_types.h
+++ linux-2.6/include/asm-m32r/kmap_types.h
@@ -21,7 +21,8 @@ D(9) KM_IRQ0,
D(10) KM_IRQ1,
D(11) KM_SOFTIRQ0,
D(12) KM_SOFTIRQ1,
-D(13) KM_TYPE_NR
+D(13) KM_CLEARPAGE,
+D(14) KM_TYPE_NR
};
#undef D
Index: linux-2.6/include/asm-mn10300/kmap_types.h
===================================================================
--- linux-2.6.orig/include/asm-mn10300/kmap_types.h
+++ linux-2.6/include/asm-mn10300/kmap_types.h
@@ -25,6 +25,7 @@ enum km_type {
KM_IRQ1,
KM_SOFTIRQ0,
KM_SOFTIRQ1,
+ KM_CLEARPAGE,
KM_TYPE_NR
};
Index: linux-2.6/include/linux/highmem.h
===================================================================
--- linux-2.6.orig/include/linux/highmem.h
+++ linux-2.6/include/linux/highmem.h
@@ -124,6 +124,18 @@ static inline void clear_highpage(struct
kunmap_atomic(kaddr, KM_USER0);
}
+static inline void sanitize_highpage(struct page *page)
+{
+ void *kaddr;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ kaddr = kmap_atomic(page, KM_CLEARPAGE);
+ clear_page(kaddr);
+ kunmap_atomic(kaddr, KM_CLEARPAGE);
+ local_irq_restore(flags);
+}
+
static inline void zero_user_segments(struct page *page,
unsigned start1, unsigned end1,
unsigned start2, unsigned end2)
Index: linux-2.6/mm/page_alloc.c
===================================================================
--- linux-2.6.orig/mm/page_alloc.c
+++ linux-2.6/mm/page_alloc.c
@@ -123,6 +123,7 @@ int min_free_kbytes = 1024;
unsigned long __meminitdata nr_kernel_pages;
unsigned long __meminitdata nr_all_pages;
static unsigned long __meminitdata dma_reserve;
+int sanitize_all_mem;
#ifdef CONFIG_ARCH_POPULATES_NODE_MAP
/*
@@ -221,6 +222,15 @@ static inline int bad_range(struct zone
}
#endif
+static __init int setup_page_sanitization(char *s)
+{
+ printk(KERN_INFO "Memory sanitization enabled.\n");
+ sanitize_all_mem = 1;
+
+ return 0;
+}
+early_param("sanitize_mem", setup_page_sanitization);
+
static void bad_page(struct page *page)
{
static unsigned long resume;
@@ -545,6 +555,7 @@ static void free_one_page(struct zone *z
static void __free_pages_ok(struct page *page, unsigned int order)
{
+ unsigned long index = 1UL << order;
unsigned long flags;
int i;
int bad = 0;
@@ -559,6 +570,16 @@ static void __free_pages_ok(struct page
debug_check_no_obj_freed(page_address(page),
PAGE_SIZE << order);
}
+
+ /*
+ * Page sanitization is enabled, let's clear the page contents before
+ * release.
+ */
+ if (sanitize_all_mem) {
+ for (; index; --index)
+ sanitize_highpage(page + index - 1);
+ }
+
arch_free_page(page, order);
kernel_map_pages(page, 1 << order, 0);
@@ -647,7 +668,7 @@ static int prep_new_page(struct page *pa
arch_alloc_page(page, order);
kernel_map_pages(page, 1 << order, 1);
- if (gfp_flags & __GFP_ZERO)
+ if ((gfp_flags & __GFP_ZERO) && !sanitize_all_mem)
prep_zero_page(page, order, gfp_flags);
if (order && (gfp_flags & __GFP_COMP))
@@ -1009,6 +1030,10 @@ static void free_hot_cold_page(struct pa
debug_check_no_locks_freed(page_address(page), PAGE_SIZE);
debug_check_no_obj_freed(page_address(page), PAGE_SIZE);
}
+
+ if (sanitize_all_mem)
+ sanitize_highpage(page);
+
arch_free_page(page, 0);
kernel_map_pages(page, 1, 0);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2009-05-23 18:22 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-05-20 18:30 [patch 0/5] Support for sanitization flag in low-level page allocator Larry H.
2009-05-20 20:42 ` Peter Zijlstra
2009-05-20 21:24 ` Larry H.
2009-05-21 15:21 ` Robin Holt
2009-05-21 18:43 ` Larry H.
2009-05-29 22:58 ` Andrew Morton
2009-05-30 7:00 ` Larry H.
2009-05-30 7:12 ` Pekka Enberg
2009-05-30 7:35 ` Larry H.
2009-05-30 7:39 ` Pekka Enberg
2009-05-21 19:08 ` Rik van Riel
2009-05-21 19:26 ` Alan Cox
2009-05-21 19:56 ` Larry H.
2009-05-21 20:47 ` Alan Cox
2009-05-21 21:46 ` Larry H.
2009-05-21 22:47 ` Alan Cox
2009-05-22 11:22 ` Larry H.
2009-05-22 13:37 ` Alan Cox
2009-05-26 19:02 ` Pavel Machek
2009-05-21 19:17 ` Rik van Riel
2009-05-21 19:30 ` Larry H.
2009-05-22 7:34 ` Ingo Molnar
2009-05-22 11:38 ` Larry H.
2009-05-22 13:39 ` Alan Cox
2009-05-22 18:03 ` Larry H.
2009-05-22 18:21 ` Alan Cox
2009-05-22 23:25 ` [PATCH] Support for kernel memory sanitization Larry H.
2009-05-22 23:52 ` Randy Dunlap
2009-05-22 23:40 ` [patch 0/5] Support for sanitization flag in low-level page allocator Larry H.
2009-05-23 8:09 ` Alan Cox
2009-05-23 15:56 ` Arjan van de Ven
2009-05-23 18:21 ` Larry H. [this message]
2009-05-23 21:05 ` [PATCH] Support for unconditional page sanitization Arjan van de Ven
2009-05-24 10:19 ` pageexec
2009-05-24 16:38 ` Arjan van de Ven
2009-05-28 19:36 ` [patch 0/5] Support for sanitization flag in low-level page allocator Peter Zijlstra
2009-05-29 14:32 ` Arjan van de Ven
2009-05-30 5:48 ` Larry H.
2009-05-30 10:39 ` Peter Zijlstra
2009-05-30 10:43 ` Larry H.
2009-05-30 11:42 ` pageexec
2009-05-30 13:21 ` Peter Zijlstra
2009-05-30 13:24 ` Peter Zijlstra
2009-05-30 13:54 ` pageexec
2009-05-30 14:04 ` Larry H.
2009-05-30 14:13 ` Rik van Riel
2009-05-30 14:08 ` Rik van Riel
2009-05-30 14:30 ` Alan Cox
2009-05-30 14:45 ` Peter Zijlstra
2009-05-30 14:48 ` Rik van Riel
2009-05-30 17:00 ` Larry H.
2009-05-30 17:25 ` Larry H.
2009-05-30 18:32 ` Ingo Molnar
2009-06-05 13:15 ` Pavel Machek
2009-05-31 14:38 ` Arjan van de Ven
2009-05-31 15:03 ` Arjan van de Ven
2009-05-22 18:37 ` Nai Xia
2009-05-22 19:18 ` Nai Xia
2009-05-23 12:49 ` Ingo Molnar
2009-05-23 22:28 ` Larry H.
2009-05-23 22:42 ` Rik van Riel
2009-05-25 1:17 ` [PATCH] Sanitize memory on kfree() and kmem_cache_free() Larry H.
2009-05-27 22:34 ` [patch 0/5] Support for sanitization flag in low-level page allocator Ingo Molnar
2009-05-28 6:27 ` Alan Cox
2009-05-28 7:00 ` Larry H.
2009-05-28 9:08 ` Ingo Molnar
2009-05-28 11:50 ` Alan Cox
2009-05-28 19:44 ` Peter Zijlstra
2009-05-30 7:35 ` Pekka Enberg
2009-05-30 7:50 ` Larry H.
2009-05-30 7:53 ` Pekka Enberg
2009-05-30 8:20 ` Larry H.
2009-05-30 8:33 ` Pekka Enberg
2009-05-30 15:05 ` Ray Lee
2009-05-30 17:34 ` Ingo Molnar
2009-05-30 18:03 ` Larry H.
2009-05-30 18:21 ` Ingo Molnar
2009-05-30 18:45 ` Larry H.
2009-05-30 19:08 ` Ingo Molnar
2009-05-30 20:39 ` Rik van Riel
2009-05-30 20:53 ` Pekka Enberg
2009-05-30 21:33 ` Larry H.
2009-05-30 23:13 ` Alan Cox
2009-05-30 23:18 ` Larry H.
2009-05-31 6:30 ` Pekka Enberg
2009-05-31 11:49 ` Larry H.
2009-05-31 7:17 ` Pekka Enberg
2009-05-31 11:58 ` Larry H.
2009-05-31 12:16 ` Pekka Enberg
2009-05-31 12:30 ` Larry H.
2009-05-31 12:35 ` Pekka Enberg
2009-05-30 23:10 ` Alan Cox
2009-05-31 6:14 ` Pekka Enberg
2009-05-31 10:24 ` Alan Cox
2009-05-31 10:24 ` Pekka Enberg
2009-05-31 12:16 ` Larry H.
2009-05-31 12:19 ` Pekka Enberg
2009-05-31 16:25 ` Alan Cox
2009-05-30 22:10 ` Ingo Molnar
2009-05-30 23:15 ` Alan Cox
2009-05-30 20:22 ` Pekka Enberg
2009-05-30 22:14 ` Ingo Molnar
2009-05-30 17:39 ` Ingo Molnar
2009-05-30 7:57 ` Pekka Enberg
2009-05-30 9:05 ` Larry H.
2009-05-30 17:46 ` Ingo Molnar
2009-05-30 18:09 ` Larry H.
2009-05-30 8:31 ` Alan Cox
2009-05-30 8:35 ` Pekka Enberg
2009-05-30 9:27 ` Larry H.
2009-05-28 18:48 ` pageexec
2009-05-30 17:50 ` Ingo Molnar
2009-05-28 12:48 ` Pavel Machek
2009-05-28 12:55 ` Larry H.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090523182141.GK13971@oblivion.subreption.com \
--to=research@subreption.com \
--cc=alan@lxorguk.ukuu.org.uk \
--cc=arjan@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@elte.hu \
--cc=mingo@redhat.com \
--cc=pageexec@freemail.hu \
--cc=riel@redhat.com \
--cc=torvalds@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox