[PATCH 0/9] skb fragment API: convert network drivers (part V)

[PATCH 0/9] skb fragment API: convert network drivers (part V)

[Ian Campbell]

The following series converts a fifth (and hopefully final) batch of
network drivers to the SKB pages fragment API introduced in
131ea6675c76.

There are four drivers here (mlx4, cxgb4, cxgb4vf and cxgbi) which used
skb_frag_t as part of their internal datastructures which meant that
they are impacted by changes to that type more than most drivers. To
break this dependency I added a "struct subpage" (struct page + offset +
len) and converted them to use it. These conversions are a little less
trivial than most of the preceding ones and I have only been able to
compile test them.

I think "struct subpage" is a generally useful tuple I added to a
central location (mm_types.h) rather than somewhere networking or driver
specific but I can trivially move if preferred.

The remaining three drivers in the series (ehea, emac, ll_temac) are
normal conversions which I either missed in my first pass or which have
had direct uses of the fragment pages added since then.

The final patch here wraps the page member of skb_frag_t in a structure,
this is a precursor to adding the destructor here (those patches need a
little more work, arising from comments made at LPC, I'll post regarding
those shortly). This should help ensure that no direct uses of the page
get introduced in the meantime.

I have run an allmodconfig build on a boatload architectures[2] on a
baseline of current net-next/master (88c5100c28b0) and with this series.
Although the baseline didn't build for most architectures I used "make
-k" and confirmed that this series added no new warnings or errors.

This is part of my series to enable visibility into SKB paged fragment's
lifecycles, [0] contains some more background and rationale but
basically the completed series will allow entities which inject pages
into the networking stack to receive a notification when the stack has
really finished with those pages (i.e. including retransmissions,
clones, pull-ups etc) and not just when the original skb is finished
with, which is beneficial to many subsystems which wish to inject pages
into the network stack without giving up full ownership of those page's
lifecycle. It implements something broadly along the lines of what was
described in [1].

Cheers,
Ian.

[0] http://marc.info/?l=linux-netdev&m=131072801125521&w=2
[1] http://marc.info/?l=linux-netdev&m=130925719513084&w=2
[2] arm amd64 blackfin cris i386 ia64 m68k mips64 mips powerpc64 powerpc
s390x sh4 sparc64 sparc xtensa 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 1/9] mm: add a "struct subpage" type containing a page, offset and length

[Ian Campbell]

A few network drivers currently use skb_frag_struct for this purpose but I have
patches which add additional fields and semantics there which these other uses
do not want.

A structure for reference sub-page regions seems like a generally useful thing
so do so instead of adding a network subsystem specific structure.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: linux-mm@kvack.org
---
 include/linux/mm_types.h |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 774b895..dc1d103 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -135,6 +135,17 @@ struct page {
 #endif
 ;
 
+struct subpage {
+	struct page *page;
+#if (BITS_PER_LONG > 32) || (PAGE_SIZE >= 65536)
+	__u32 page_offset;
+	__u32 size;
+#else
+	__u16 page_offset;
+	__u16 size;
+#endif
+};
+
 typedef unsigned long __nocast vm_flags_t;
 
 /*
-- 
1.7.2.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/9] mm: add a "struct subpage" type containing a page, offset and length

[Ian Campbell]

On Mon, 2011-10-10 at 12:11 +0100, Ian Campbell wrote:

> Cc: linux-mm@kvack.org
> ---
>  include/linux/mm_types.h |   11 +++++++++++
>  1 files changed, 11 insertions(+), 0 deletions(-)

get_maintainers.pl didn't pick up on this CC. Since mm_types.h was split
out of mm.h does the following make sense? Not sure if mm_*.h (or just
mm_inline.hm?) also makes sense.

8<--------------------------

Subject: MAINTAINER: mm subsystem includes mm_types.h

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff --git a/MAINTAINERS b/MAINTAINERS
index ae8820e..f10a7ea 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4212,6 +4212,7 @@ L:	linux-mm@kvack.org
 W:	http://www.linux-mm.org
 S:	Maintained
 F:	include/linux/mm.h
+F:	include/linux/mm_types.h
 F:	mm/
 
 MEMORY RESOURCE CONTROLLER


> 
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 774b895..dc1d103 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -135,6 +135,17 @@ struct page {
>  #endif
>  ;
>  
> +struct subpage {
> +	struct page *page;
> +#if (BITS_PER_LONG > 32) || (PAGE_SIZE >= 65536)
> +	__u32 page_offset;
> +	__u32 size;
> +#else
> +	__u16 page_offset;
> +	__u16 size;
> +#endif
> +};
> +
>  typedef unsigned long __nocast vm_flags_t;
>  
>  /*


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/9] mm: add a "struct subpage" type containing a page, offset and length

[Christoph Hellwig]

On Mon, Oct 10, 2011 at 12:11:33PM +0100, Ian Campbell wrote:
> A few network drivers currently use skb_frag_struct for this purpose but I have
> patches which add additional fields and semantics there which these other uses
> do not want.
> 
> A structure for reference sub-page regions seems like a generally useful thing
> so do so instead of adding a network subsystem specific structure.

Subpage seems like a fairly bad name.  page_frag would fit into the
scheme used in a few other places.

The brings back the discussion of unifying the various incarnations we
have of this (biovec, skb frag and there were a few more at times),
but IIRC one of the sticking points back then was that one offset
insistet in 32-bit offset/len and the other on 16-bit.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/9] mm: add a "struct subpage" type containing a page, offset and length

[Ian Campbell]

On Mon, 2011-10-10 at 16:55 +0100, Christoph Hellwig wrote:
> On Mon, Oct 10, 2011 at 12:11:33PM +0100, Ian Campbell wrote:
> > A few network drivers currently use skb_frag_struct for this purpose but I have
> > patches which add additional fields and semantics there which these other uses
> > do not want.
> > 
> > A structure for reference sub-page regions seems like a generally useful thing
> > so do so instead of adding a network subsystem specific structure.
> 
> Subpage seems like a fairly bad name.  page_frag would fit into the
> scheme used in a few other places.

ok.

> The brings back the discussion of unifying the various incarnations we
> have of this (biovec, skb frag and there were a few more at times),
> but IIRC one of the sticking points back then was that one offset
> insistet in 32-bit offset/len and the other on 16-bit.

This version sizes the fields according to page size, was there
somewhere which wanted to use an offset > PAGE_SIZE (or size > PAGE_SIZE
for that matter). That would be pretty odd and/or not really a candidate
for using this datastructure?

Ian.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/9] mm: add a "struct subpage" type containing a page, offset and length

[Christoph Hellwig]

On Mon, Oct 10, 2011 at 05:10:59PM +0100, Ian Campbell wrote:
> This version sizes the fields according to page size, was there
> somewhere which wanted to use an offset > PAGE_SIZE (or size > PAGE_SIZE
> for that matter). That would be pretty odd and/or not really a candidate
> for using this datastructure?

I wasn't ever part of the fight myself and only vaguely remember it.
Try to get linux-kernel and Jens onto the Cc list to at least have the
major stakeholders informed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/9] mm: add a "struct subpage" type containing a page, offset and length

[Ian Campbell]

(reposting including LKML to catch other potential users)

Is this structure of any use to unify other instances of a similar
tuple, e.g. biovec, pagefrag etc?

Ian.

On Mon, 2011-10-10 at 12:11 +0100, Ian Campbell wrote:
> A few network drivers currently use skb_frag_struct for this purpose but I have
> patches which add additional fields and semantics there which these other uses
> do not want.
> 
> A structure for reference sub-page regions seems like a generally useful thing
> so do so instead of adding a network subsystem specific structure.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: linux-mm@kvack.org
> ---
>  include/linux/mm_types.h |   11 +++++++++++
>  1 files changed, 11 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 774b895..dc1d103 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -135,6 +135,17 @@ struct page {
>  #endif
>  ;
>  
> +struct subpage {
> +	struct page *page;
> +#if (BITS_PER_LONG > 32) || (PAGE_SIZE >= 65536)
> +	__u32 page_offset;
> +	__u32 size;
> +#else
> +	__u16 page_offset;
> +	__u16 size;
> +#endif
> +};
> +
>  typedef unsigned long __nocast vm_flags_t;
>  
>  /*


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/9] skb fragment API: convert network drivers (part V)

[David Miller]

From: Ian Campbell <Ian.Campbell@citrix.com>
Date: Mon, 10 Oct 2011 12:11:16 +0100

> I think "struct subpage" is a generally useful tuple I added to a
> central location (mm_types.h) rather than somewhere networking or driver
> specific but I can trivially move if preferred.

I'm fine with the patch series, but this generic datastructure
addition needs some feedback first.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/9] skb fragment API: convert network drivers (part V)

[Eric Dumazet]

Le lundi 10 octobre 2011 A  14:20 -0400, David Miller a A(C)crit :
> From: Ian Campbell <Ian.Campbell@citrix.com>
> Date: Mon, 10 Oct 2011 12:11:16 +0100
> 
> > I think "struct subpage" is a generally useful tuple I added to a
> > central location (mm_types.h) rather than somewhere networking or driver
> > specific but I can trivially move if preferred.
> 
> I'm fine with the patch series, but this generic datastructure
> addition needs some feedback first.

I was planning to send a patch to abstract frag->size manipulation and
ease upcoming truesize certification work.

static inline int skb_frag_size(const skb_frag_t *frag)
{
	return frag->size;
}

static inline void skb_frag_size_set(skb_frag_t *frag, int size)
{
	frag->size = size;
}

static inline void skb_frag_size_add(skb_frag_t *frag, int size)
{
	frag->size += size;
}

static inline void skb_frag_size_sub(skb_frag_t *frag, int size)
{
	frag->size -= size;
}

Is it OK if I send a single patch right now ?

I am asking because it might clash a bit with Ian work.


 drivers/atm/eni.c                                    |    2 
 drivers/infiniband/hw/amso1100/c2.c                  |    4 
 drivers/infiniband/hw/nes/nes_nic.c                  |   10 -
 drivers/infiniband/ulp/ipoib/ipoib_cm.c              |    2 
 drivers/infiniband/ulp/ipoib/ipoib_ib.c              |   18 +-
 drivers/net/ethernet/3com/3c59x.c                    |    6 
 drivers/net/ethernet/3com/typhoon.c                  |    6 
 drivers/net/ethernet/adaptec/starfire.c              |    8 -
 drivers/net/ethernet/aeroflex/greth.c                |    8 -
 drivers/net/ethernet/alteon/acenic.c                 |   10 -
 drivers/net/ethernet/atheros/atl1c/atl1c_main.c      |    2 
 drivers/net/ethernet/atheros/atl1e/atl1e_main.c      |    6 
 drivers/net/ethernet/atheros/atlx/atl1.c             |   12 -
 drivers/net/ethernet/broadcom/bnx2.c                 |   12 -
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c      |   14 -
 drivers/net/ethernet/broadcom/tg3.c                  |    8 -
 drivers/net/ethernet/brocade/bna/bnad.c              |    6 
 drivers/net/ethernet/chelsio/cxgb/sge.c              |   10 -
 drivers/net/ethernet/chelsio/cxgb3/sge.c             |   12 -
 drivers/net/ethernet/chelsio/cxgb4/sge.c             |   26 +--
 drivers/net/ethernet/chelsio/cxgb4vf/sge.c           |   26 +--
 drivers/net/ethernet/cisco/enic/enic_main.c          |   12 -
 drivers/net/ethernet/emulex/benet/be_main.c          |   18 +-
 drivers/net/ethernet/ibm/ehea/ehea_main.c            |    8 -
 drivers/net/ethernet/ibm/emac/core.c                 |    2 
 drivers/net/ethernet/ibm/ibmveth.c                   |    6 
 drivers/net/ethernet/intel/e1000/e1000_main.c        |    6 
 drivers/net/ethernet/intel/e1000e/netdev.c           |    6 
 drivers/net/ethernet/intel/igb/igb_main.c            |    2 
 drivers/net/ethernet/intel/igbvf/netdev.c            |    4 
 drivers/net/ethernet/intel/ixgb/ixgb_main.c          |    4 
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c        |    4 
 drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c    |    6 
 drivers/net/ethernet/jme.c                           |    4 
 drivers/net/ethernet/marvell/mv643xx_eth.c           |    9 -
 drivers/net/ethernet/marvell/skge.c                  |    8 -
 drivers/net/ethernet/marvell/sky2.c                  |   16 +-
 drivers/net/ethernet/mellanox/mlx4/en_rx.c           |   14 -
 drivers/net/ethernet/mellanox/mlx4/en_tx.c           |   12 -
 drivers/net/ethernet/micrel/ksz884x.c                |    2 
 drivers/net/ethernet/myricom/myri10ge/myri10ge.c     |   14 -
 drivers/net/ethernet/natsemi/ns83820.c               |    4 
 drivers/net/ethernet/neterion/s2io.c                 |   12 -
 drivers/net/ethernet/neterion/vxge/vxge-main.c       |   12 -
 drivers/net/ethernet/nvidia/forcedeth.c              |   18 +-
 drivers/net/ethernet/pasemi/pasemi_mac.c             |    8 -
 drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c |    6 
 drivers/net/ethernet/qlogic/qla3xxx.c                |    6 
 drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c     |    6 
 drivers/net/ethernet/qlogic/qlge/qlge_main.c         |    6 
 drivers/net/ethernet/realtek/8139cp.c                |    4 
 drivers/net/ethernet/realtek/r8169.c                 |    4 
 drivers/net/ethernet/sfc/rx.c                        |    2 
 drivers/net/ethernet/sfc/tx.c                        |    8 -
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c    |    4 
 drivers/net/ethernet/sun/cassini.c                   |    8 -
 drivers/net/ethernet/sun/niu.c                       |    6 
 drivers/net/ethernet/sun/sungem.c                    |    4 
 drivers/net/ethernet/sun/sunhme.c                    |    4 
 drivers/net/ethernet/tehuti/tehuti.c                 |    6 
 drivers/net/ethernet/tile/tilepro.c                  |    2 
 drivers/net/ethernet/tundra/tsi108_eth.c             |    6 
 drivers/net/ethernet/via/via-velocity.c              |    6 
 drivers/net/ethernet/xilinx/ll_temac_main.c          |    4 
 drivers/net/virtio_net.c                             |    8 -
 drivers/net/vmxnet3/vmxnet3_drv.c                    |   12 -
 drivers/net/xen-netback/netback.c                    |    4 
 drivers/net/xen-netfront.c                           |    4 
 drivers/scsi/cxgbi/libcxgbi.c                        |   10 -
 drivers/scsi/fcoe/fcoe_transport.c                   |    2 
 drivers/staging/hv/netvsc_drv.c                      |    4 
 include/linux/skbuff.h                               |   28 +++
 net/appletalk/ddp.c                                  |    5 
 net/core/datagram.c                                  |   16 +-
 net/core/dev.c                                       |    6 
 net/core/pktgen.c                                    |   12 -
 net/core/skbuff.c                                    |   72 +++++-----
 net/core/user_dma.c                                  |    4 
 net/ipv4/inet_lro.c                                  |    8 -
 net/ipv4/ip_fragment.c                               |    4 
 net/ipv4/ip_output.c                                 |    6 
 net/ipv4/tcp.c                                       |    9 -
 net/ipv4/tcp_output.c                                |    8 -
 net/ipv6/ip6_output.c                                |    5 
 net/ipv6/netfilter/nf_conntrack_reasm.c              |    4 
 net/ipv6/reassembly.c                                |    4 
 net/xfrm/xfrm_ipcomp.c                               |    2 
 87 files changed, 389 insertions(+), 359 deletions(-)


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/9] skb fragment API: convert network drivers (part V)

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Mon, 10 Oct 2011 20:52:11 +0200

> Is it OK if I send a single patch right now ?
> 
> I am asking because it might clash a bit with Ian work.

Feel free to do so, we'll sort it out somehow.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/9] skb fragment API: convert network drivers (part V)

[Ian Campbell]

On Mon, 2011-10-10 at 19:52 +0100, Eric Dumazet wrote:
> Le lundi 10 octobre 2011 a 14:20 -0400, David Miller a ecrit :
> > From: Ian Campbell <Ian.Campbell@citrix.com>
> > Date: Mon, 10 Oct 2011 12:11:16 +0100
> > 
> > > I think "struct subpage" is a generally useful tuple I added to a
> > > central location (mm_types.h) rather than somewhere networking or driver
> > > specific but I can trivially move if preferred.
> > 
> > I'm fine with the patch series, but this generic datastructure
> > addition needs some feedback first.

Sure. Would you take patches 6, 7 & 8 now? They don't rely on the new
struct.

> I was planning to send a patch to abstract frag->size manipulation and
> ease upcoming truesize certification work.
[...]
> Is it OK if I send a single patch right now ?
> 
> I am asking because it might clash a bit with Ian work.

FWIW it's fine with me, there is only the half dozen or so drivers in
this series left to convert and I can rebase pretty easily.

Ian.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/9] skb fragment API: convert network drivers (part V)

[David Miller]

From: Ian Campbell <Ian.Campbell@citrix.com>
Date: Mon, 10 Oct 2011 20:17:04 +0100

> On Mon, 2011-10-10 at 19:52 +0100, Eric Dumazet wrote:
>> Le lundi 10 octobre 2011 à 14:20 -0400, David Miller a écrit :
>> > From: Ian Campbell <Ian.Campbell@citrix.com>
>> > Date: Mon, 10 Oct 2011 12:11:16 +0100
>> > 
>> > > I think "struct subpage" is a generally useful tuple I added to a
>> > > central location (mm_types.h) rather than somewhere networking or driver
>> > > specific but I can trivially move if preferred.
>> > 
>> > I'm fine with the patch series, but this generic datastructure
>> > addition needs some feedback first.
> 
> Sure. Would you take patches 6, 7 & 8 now? They don't rely on the new
> struct.

I'll do that right now, thanks Ian.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/9] skb fragment API: convert network drivers (part V)

[Eric Dumazet]

Le lundi 10 octobre 2011 A  15:18 -0400, David Miller a A(C)crit :
> From: Ian Campbell <Ian.Campbell@citrix.com>
> Date: Mon, 10 Oct 2011 20:17:04 +0100
> 
> > On Mon, 2011-10-10 at 19:52 +0100, Eric Dumazet wrote:
> >> Le lundi 10 octobre 2011 A  14:20 -0400, David Miller a A(C)crit :
> >> > From: Ian Campbell <Ian.Campbell@citrix.com>
> >> > Date: Mon, 10 Oct 2011 12:11:16 +0100
> >> > 
> >> > > I think "struct subpage" is a generally useful tuple I added to a
> >> > > central location (mm_types.h) rather than somewhere networking or driver
> >> > > specific but I can trivially move if preferred.
> >> > 
> >> > I'm fine with the patch series, but this generic datastructure
> >> > addition needs some feedback first.
> > 
> > Sure. Would you take patches 6, 7 & 8 now? They don't rely on the new
> > struct.
> 
> I'll do that right now, thanks Ian.

I'll respin my patch once your tree is pushed.

Thanks


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>