* [PATCHv2 0/2] vhost: a kernel-level virtio server @ 2009-08-11 21:27 Michael S. Tsirkin 2009-08-11 23:49 ` Gregory Haskins 0 siblings, 1 reply; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-11 21:27 UTC (permalink / raw) To: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm, hpa This implements vhost: a kernel-level backend for virtio, The main motivation for this work is to reduce virtualization overhead for virtio by removing system calls on data path, without guest changes. For virtio-net, this removes up to 4 system calls per packet: vm exit for kick, reentry for kick, iothread wakeup for packet, interrupt injection for packet. Some more detailed description attached to the patch itself. The patches are against 2.6.31-rc4. I'd like them to go into linux-next and down the road 2.6.32 if possible. Please comment. Changes from v1: - Move use_mm/unuse_mm from fs/aio.c to mm instead of copying. - Reorder code to avoid need for forward declarations - Kill a couple of debugging printks Michael S. Tsirkin (2): mm: export use_mm/unuse_mm to modules vhost_net: a kernel-level virtio server MAINTAINERS | 10 + arch/x86/kvm/Kconfig | 1 + drivers/Makefile | 1 + drivers/vhost/Kconfig | 11 + drivers/vhost/Makefile | 2 + drivers/vhost/net.c | 411 +++++++++++++++++++++++++++ drivers/vhost/vhost.c | 663 +++++++++++++++++++++++++++++++++++++++++++ drivers/vhost/vhost.h | 108 +++++++ fs/aio.c | 47 +--- include/linux/Kbuild | 1 + include/linux/miscdevice.h | 1 + include/linux/mmu_context.h | 9 + include/linux/vhost.h | 100 +++++++ mm/Makefile | 2 +- mm/mmu_context.c | 58 ++++ 15 files changed, 1378 insertions(+), 47 deletions(-) create mode 100644 drivers/vhost/Kconfig create mode 100644 drivers/vhost/Makefile create mode 100644 drivers/vhost/net.c create mode 100644 drivers/vhost/vhost.c create mode 100644 drivers/vhost/vhost.h create mode 100644 include/linux/mmu_context.h create mode 100644 include/linux/vhost.h create mode 100644 mm/mmu_context.c -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-11 21:27 [PATCHv2 0/2] vhost: a kernel-level virtio server Michael S. Tsirkin @ 2009-08-11 23:49 ` Gregory Haskins 2009-08-12 7:16 ` Michael S. Tsirkin 0 siblings, 1 reply; 15+ messages in thread From: Gregory Haskins @ 2009-08-11 23:49 UTC (permalink / raw) To: Michael S. Tsirkin Cc: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm, hpa [-- Attachment #1: Type: text/plain, Size: 758 bytes --] Michael S. Tsirkin wrote: > This implements vhost: a kernel-level backend for virtio, > The main motivation for this work is to reduce virtualization > overhead for virtio by removing system calls on data path, > without guest changes. For virtio-net, this removes up to > 4 system calls per packet: vm exit for kick, reentry for kick, > iothread wakeup for packet, interrupt injection for packet. > > Some more detailed description attached to the patch itself. > > The patches are against 2.6.31-rc4. I'd like them to go into linux-next > and down the road 2.6.32 if possible. Please comment. I will add this series to my benchmark run in the next day or so. Any specific instructions on how to set it up and run? Regards, -Greg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 267 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-11 23:49 ` Gregory Haskins @ 2009-08-12 7:16 ` Michael S. Tsirkin 2009-08-12 11:56 ` Gregory Haskins 0 siblings, 1 reply; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 7:16 UTC (permalink / raw) To: Gregory Haskins Cc: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm, hpa On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > This implements vhost: a kernel-level backend for virtio, > > The main motivation for this work is to reduce virtualization > > overhead for virtio by removing system calls on data path, > > without guest changes. For virtio-net, this removes up to > > 4 system calls per packet: vm exit for kick, reentry for kick, > > iothread wakeup for packet, interrupt injection for packet. > > > > Some more detailed description attached to the patch itself. > > > > The patches are against 2.6.31-rc4. I'd like them to go into linux-next > > and down the road 2.6.32 if possible. Please comment. > > I will add this series to my benchmark run in the next day or so. Any > specific instructions on how to set it up and run? > > Regards, > -Greg > 1. use a dedicated network interface with SRIOV, program mac to match that of guest (for testing, you can set promisc mode, but that is bad for performance) 2. disable tso,gso,lro with ethtool 3. add vhost=ethX -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 7:16 ` Michael S. Tsirkin @ 2009-08-12 11:56 ` Gregory Haskins 2009-08-12 12:05 ` Michael S. Tsirkin 0 siblings, 1 reply; 15+ messages in thread From: Gregory Haskins @ 2009-08-12 11:56 UTC (permalink / raw) To: Michael S. Tsirkin Cc: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm@linux-foundation.org >> Andrew Morton, hpa, Patrick Mullaney [-- Attachment #1: Type: text/plain, Size: 1590 bytes --] Michael S. Tsirkin wrote: > On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> This implements vhost: a kernel-level backend for virtio, >>> The main motivation for this work is to reduce virtualization >>> overhead for virtio by removing system calls on data path, >>> without guest changes. For virtio-net, this removes up to >>> 4 system calls per packet: vm exit for kick, reentry for kick, >>> iothread wakeup for packet, interrupt injection for packet. >>> >>> Some more detailed description attached to the patch itself. >>> >>> The patches are against 2.6.31-rc4. I'd like them to go into linux-next >>> and down the road 2.6.32 if possible. Please comment. >> I will add this series to my benchmark run in the next day or so. Any >> specific instructions on how to set it up and run? >> >> Regards, >> -Greg >> > > 1. use a dedicated network interface with SRIOV, program mac to match > that of guest (for testing, you can set promisc mode, but that is > bad for performance) Are you saying SRIOV is a requirement, and I can either program the SRIOV adapter with a mac or use promis? Or are you saying I can use SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > 2. disable tso,gso,lro with ethtool Out of curiosity, wouldnt you only need to disable LRO on the adapter, since the other two (IIUC) are transmit path and are therefore influenced by the skb's you generate in vhost? > 3. add vhost=ethX You mean via "ip link" I assume? Regards, -Greg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 267 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 11:56 ` Gregory Haskins @ 2009-08-12 12:05 ` Michael S. Tsirkin 2009-08-12 12:41 ` Gregory Haskins 0 siblings, 1 reply; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 12:05 UTC (permalink / raw) To: Gregory Haskins Cc: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm@linux-foundation.org >> Andrew Morton, hpa, Patrick Mullaney On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: > >> Michael S. Tsirkin wrote: > >>> This implements vhost: a kernel-level backend for virtio, > >>> The main motivation for this work is to reduce virtualization > >>> overhead for virtio by removing system calls on data path, > >>> without guest changes. For virtio-net, this removes up to > >>> 4 system calls per packet: vm exit for kick, reentry for kick, > >>> iothread wakeup for packet, interrupt injection for packet. > >>> > >>> Some more detailed description attached to the patch itself. > >>> > >>> The patches are against 2.6.31-rc4. I'd like them to go into linux-next > >>> and down the road 2.6.32 if possible. Please comment. > >> I will add this series to my benchmark run in the next day or so. Any > >> specific instructions on how to set it up and run? > >> > >> Regards, > >> -Greg > >> > > > > 1. use a dedicated network interface with SRIOV, program mac to match > > that of guest (for testing, you can set promisc mode, but that is > > bad for performance) > > Are you saying SRIOV is a requirement, and I can either program the > SRIOV adapter with a mac or use promis? Or are you saying I can use > SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). SRIOV is not a requirement. And you can also use a dedicated nic+programmed mac if you are so inclined. > > 2. disable tso,gso,lro with ethtool > > Out of curiosity, wouldnt you only need to disable LRO on the adapter, > since the other two (IIUC) are transmit path and are therefore > influenced by the skb's you generate in vhost? Hmm, makes sense. I'll check this and let you know. > > > 3. add vhost=ethX > > You mean via "ip link" I assume? No, that's a new flag for virtio in qemu: -net nic,model=virtio,vhost=veth0 > Regards, > -Greg > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 12:05 ` Michael S. Tsirkin @ 2009-08-12 12:41 ` Gregory Haskins 2009-08-12 12:52 ` Arnd Bergmann 2009-08-12 13:04 ` Michael S. Tsirkin 0 siblings, 2 replies; 15+ messages in thread From: Gregory Haskins @ 2009-08-12 12:41 UTC (permalink / raw) To: Michael S. Tsirkin Cc: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm@linux-foundation.org >> Andrew Morton, hpa, Patrick Mullaney [-- Attachment #1: Type: text/plain, Size: 1049 bytes --] Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: <snip> >>> >>> 1. use a dedicated network interface with SRIOV, program mac to match >>> that of guest (for testing, you can set promisc mode, but that is >>> bad for performance) >> >> Are you saying SRIOV is a requirement, and I can either program the >> SRIOV adapter with a mac or use promis? Or are you saying I can use >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > SRIOV is not a requirement. And you can also use a dedicated > nic+programmed mac if you are so inclined. Makes sense. Got it. I was going to add guest-to-guest to the test matrix, but I assume that is not supported with vhost unless you have something like a VEPA enabled bridge? <snip> >>> 3. add vhost=ethX >> You mean via "ip link" I assume? > > No, that's a new flag for virtio in qemu: > > -net nic,model=virtio,vhost=veth0 Ah, ok. Even better. Thanks! -Greg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 267 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 12:41 ` Gregory Haskins @ 2009-08-12 12:52 ` Arnd Bergmann 2009-08-12 13:06 ` Michael S. Tsirkin 2009-08-12 13:04 ` Michael S. Tsirkin 1 sibling, 1 reply; 15+ messages in thread From: Arnd Bergmann @ 2009-08-12 12:52 UTC (permalink / raw) To: Gregory Haskins Cc: Michael S. Tsirkin, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney On Wednesday 12 August 2009, Gregory Haskins wrote: > >> Are you saying SRIOV is a requirement, and I can either program the > >> SRIOV adapter with a mac or use promis? Or are you saying I can use > >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > > > SRIOV is not a requirement. And you can also use a dedicated > > nic+programmed mac if you are so inclined. > > Makes sense. Got it. > > I was going to add guest-to-guest to the test matrix, but I assume that > is not supported with vhost unless you have something like a VEPA > enabled bridge? > If I understand it correctly, you can at least connect a veth pair to a bridge, right? Something like veth0 - veth1 - vhost - guest 1 eth0 - br0-| veth2 - veth3 - vhost - guest 2 It's a bit more complicated than it need to be, but should work fine. Arnd <>< -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 12:52 ` Arnd Bergmann @ 2009-08-12 13:06 ` Michael S. Tsirkin 2009-08-12 13:40 ` Arnd Bergmann 0 siblings, 1 reply; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 13:06 UTC (permalink / raw) To: Arnd Bergmann Cc: Gregory Haskins, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney On Wed, Aug 12, 2009 at 02:52:01PM +0200, Arnd Bergmann wrote: > On Wednesday 12 August 2009, Gregory Haskins wrote: > > >> Are you saying SRIOV is a requirement, and I can either program the > > >> SRIOV adapter with a mac or use promis? Or are you saying I can use > > >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > > > > > SRIOV is not a requirement. And you can also use a dedicated > > > nic+programmed mac if you are so inclined. > > > > Makes sense. Got it. > > > > I was going to add guest-to-guest to the test matrix, but I assume that > > is not supported with vhost unless you have something like a VEPA > > enabled bridge? > > > > If I understand it correctly, you can at least connect a veth pair > to a bridge, right? Something like > > veth0 - veth1 - vhost - guest 1 > eth0 - br0-| > veth2 - veth3 - vhost - guest 2 > > It's a bit more complicated than it need to be, but should work fine. > > Arnd <>< Heh, you don't need a bridge in this picture: guest 1 - vhost - veth0 - veth1 - vhost guest 2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 13:06 ` Michael S. Tsirkin @ 2009-08-12 13:40 ` Arnd Bergmann 2009-08-12 13:42 ` Michael S. Tsirkin 2009-08-12 13:51 ` Gregory Haskins 0 siblings, 2 replies; 15+ messages in thread From: Arnd Bergmann @ 2009-08-12 13:40 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Gregory Haskins, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > > If I understand it correctly, you can at least connect a veth pair > > to a bridge, right? Something like > > > > veth0 - veth1 - vhost - guest 1 > > eth0 - br0-| > > veth2 - veth3 - vhost - guest 2 > > > Heh, you don't need a bridge in this picture: > > guest 1 - vhost - veth0 - veth1 - vhost guest 2 Sure, but the setup I described is the one that I would expect to see in practice because it gives you external connectivity. Measuring two guests communicating over a veth pair is interesting for finding the bottlenecks, but of little practical relevance. Arnd <>< -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 13:40 ` Arnd Bergmann @ 2009-08-12 13:42 ` Michael S. Tsirkin 2009-08-12 13:51 ` Gregory Haskins 1 sibling, 0 replies; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 13:42 UTC (permalink / raw) To: Arnd Bergmann Cc: Gregory Haskins, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney On Wed, Aug 12, 2009 at 03:40:44PM +0200, Arnd Bergmann wrote: > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > > > If I understand it correctly, you can at least connect a veth pair > > > to a bridge, right? Something like > > > > > > veth0 - veth1 - vhost - guest 1 > > > eth0 - br0-| > > > veth2 - veth3 - vhost - guest 2 > > > > > Heh, you don't need a bridge in this picture: > > > > guest 1 - vhost - veth0 - veth1 - vhost guest 2 > > Sure, but the setup I described is the one that I would expect > to see in practice because it gives you external connectivity. > > Measuring two guests communicating over a veth pair is > interesting for finding the bottlenecks, but of little > practical relevance. > > Arnd <>< Oh, hopefully macvlan will soon allow that. -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 13:40 ` Arnd Bergmann 2009-08-12 13:42 ` Michael S. Tsirkin @ 2009-08-12 13:51 ` Gregory Haskins 2009-08-12 14:02 ` Michael S. Tsirkin 1 sibling, 1 reply; 15+ messages in thread From: Gregory Haskins @ 2009-08-12 13:51 UTC (permalink / raw) To: Arnd Bergmann Cc: Michael S. Tsirkin, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney [-- Attachment #1: Type: text/plain, Size: 813 bytes --] Arnd Bergmann wrote: > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: >>> If I understand it correctly, you can at least connect a veth pair >>> to a bridge, right? Something like >>> >>> veth0 - veth1 - vhost - guest 1 >>> eth0 - br0-| >>> veth2 - veth3 - vhost - guest 2 >>> >> Heh, you don't need a bridge in this picture: >> >> guest 1 - vhost - veth0 - veth1 - vhost guest 2 > > Sure, but the setup I described is the one that I would expect > to see in practice because it gives you external connectivity. > > Measuring two guests communicating over a veth pair is > interesting for finding the bottlenecks, but of little > practical relevance. > > Arnd <>< Yeah, this would be the config I would be interested in. Regards, -Greg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 267 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 13:51 ` Gregory Haskins @ 2009-08-12 14:02 ` Michael S. Tsirkin 2009-08-12 16:13 ` Gregory Haskins 0 siblings, 1 reply; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 14:02 UTC (permalink / raw) To: Gregory Haskins Cc: Arnd Bergmann, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: > Arnd Bergmann wrote: > > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > >>> If I understand it correctly, you can at least connect a veth pair > >>> to a bridge, right? Something like > >>> > >>> veth0 - veth1 - vhost - guest 1 > >>> eth0 - br0-| > >>> veth2 - veth3 - vhost - guest 2 > >>> > >> Heh, you don't need a bridge in this picture: > >> > >> guest 1 - vhost - veth0 - veth1 - vhost guest 2 > > > > Sure, but the setup I described is the one that I would expect > > to see in practice because it gives you external connectivity. > > > > Measuring two guests communicating over a veth pair is > > interesting for finding the bottlenecks, but of little > > practical relevance. > > > > Arnd <>< > > Yeah, this would be the config I would be interested in. Hmm, this wouldn't be the config to use for the benchmark though: there are just too many variables. If you want both guest to guest and guest to host, create 2 nics in the guest. Here's one way to do this: -net nic,model=virtio,vlan=0 -net user,vlan=0 -net nic,vlan=1,model=virtio,vhost=veth0 -redir tcp:8022::22 -net nic,model=virtio,vlan=0 -net user,vlan=0 -net nic,vlan=1,model=virtio,vhost=veth1 -redir tcp:8023::22 In guests, for simplicity, configure eth1 and eth0 to use separate subnets. Long term, I hope macvlan will be extended to support guest to guest. > Regards, > -Greg > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 14:02 ` Michael S. Tsirkin @ 2009-08-12 16:13 ` Gregory Haskins 2009-08-12 16:37 ` Michael S. Tsirkin 0 siblings, 1 reply; 15+ messages in thread From: Gregory Haskins @ 2009-08-12 16:13 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Arnd Bergmann, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney [-- Attachment #1: Type: text/plain, Size: 1805 bytes --] Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: >> Arnd Bergmann wrote: >>> On Wednesday 12 August 2009, Michael S. Tsirkin wrote: >>>>> If I understand it correctly, you can at least connect a veth pair >>>>> to a bridge, right? Something like >>>>> >>>>> veth0 - veth1 - vhost - guest 1 >>>>> eth0 - br0-| >>>>> veth2 - veth3 - vhost - guest 2 >>>>> >>>> Heh, you don't need a bridge in this picture: >>>> >>>> guest 1 - vhost - veth0 - veth1 - vhost guest 2 >>> Sure, but the setup I described is the one that I would expect >>> to see in practice because it gives you external connectivity. >>> >>> Measuring two guests communicating over a veth pair is >>> interesting for finding the bottlenecks, but of little >>> practical relevance. >>> >>> Arnd <>< >> Yeah, this would be the config I would be interested in. > > Hmm, this wouldn't be the config to use for the benchmark though: there > are just too many variables. If you want both guest to guest and guest > to host, create 2 nics in the guest. > > Here's one way to do this: > > -net nic,model=virtio,vlan=0 -net user,vlan=0 > -net nic,vlan=1,model=virtio,vhost=veth0 > -redir tcp:8022::22 > > -net nic,model=virtio,vlan=0 -net user,vlan=0 > -net nic,vlan=1,model=virtio,vhost=veth1 > -redir tcp:8023::22 > > In guests, for simplicity, configure eth1 and eth0 > to use separate subnets. I can try to do a few variations, but what I am interested is in performance in a real-world L2 configuration. This would generally mean all hosts (virtual or physical) in the same L2 domain. If I get a chance, though, I will try to also wire them up in isolation as another data point. Regards, -Greg [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 267 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 16:13 ` Gregory Haskins @ 2009-08-12 16:37 ` Michael S. Tsirkin 0 siblings, 0 replies; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 16:37 UTC (permalink / raw) To: Gregory Haskins Cc: Arnd Bergmann, netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, Andrew Morton, hpa, Patrick Mullaney On Wed, Aug 12, 2009 at 12:13:43PM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: > >> Arnd Bergmann wrote: > >>> On Wednesday 12 August 2009, Michael S. Tsirkin wrote: > >>>>> If I understand it correctly, you can at least connect a veth pair > >>>>> to a bridge, right? Something like > >>>>> > >>>>> veth0 - veth1 - vhost - guest 1 > >>>>> eth0 - br0-| > >>>>> veth2 - veth3 - vhost - guest 2 > >>>>> > >>>> Heh, you don't need a bridge in this picture: > >>>> > >>>> guest 1 - vhost - veth0 - veth1 - vhost guest 2 > >>> Sure, but the setup I described is the one that I would expect > >>> to see in practice because it gives you external connectivity. > >>> > >>> Measuring two guests communicating over a veth pair is > >>> interesting for finding the bottlenecks, but of little > >>> practical relevance. > >>> > >>> Arnd <>< > >> Yeah, this would be the config I would be interested in. > > > > Hmm, this wouldn't be the config to use for the benchmark though: there > > are just too many variables. If you want both guest to guest and guest > > to host, create 2 nics in the guest. > > > > Here's one way to do this: > > > > -net nic,model=virtio,vlan=0 -net user,vlan=0 > > -net nic,vlan=1,model=virtio,vhost=veth0 > > -redir tcp:8022::22 > > > > -net nic,model=virtio,vlan=0 -net user,vlan=0 > > -net nic,vlan=1,model=virtio,vhost=veth1 > > -redir tcp:8023::22 > > > > In guests, for simplicity, configure eth1 and eth0 > > to use separate subnets. > > I can try to do a few variations, but what I am interested is in > performance in a real-world L2 configuration. This would generally mean > all hosts (virtual or physical) in the same L2 domain. > > If I get a chance, though, I will try to also wire them up in isolation > as another data point. > > Regards, > -Greg > > Or patch macvlan to support guest to guest: http://markmail.org/message/sjy74g57qsvdo2wh That patch needs to be updated to support guest to guest multiast, but it seems functional enough for your purposes. -- MST -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCHv2 0/2] vhost: a kernel-level virtio server 2009-08-12 12:41 ` Gregory Haskins 2009-08-12 12:52 ` Arnd Bergmann @ 2009-08-12 13:04 ` Michael S. Tsirkin 1 sibling, 0 replies; 15+ messages in thread From: Michael S. Tsirkin @ 2009-08-12 13:04 UTC (permalink / raw) To: Gregory Haskins Cc: netdev, virtualization, kvm, linux-kernel, mingo, linux-mm, akpm@linux-foundation.org >> Andrew Morton, hpa, Patrick Mullaney On Wed, Aug 12, 2009 at 08:41:31AM -0400, Gregory Haskins wrote: > Michael S. Tsirkin wrote: > > On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: > >> Michael S. Tsirkin wrote: > > <snip> > > >>> > >>> 1. use a dedicated network interface with SRIOV, program mac to match > >>> that of guest (for testing, you can set promisc mode, but that is > >>> bad for performance) > >> > >> Are you saying SRIOV is a requirement, and I can either program the > >> SRIOV adapter with a mac or use promis? Or are you saying I can use > >> SRIOV+programmed mac OR a regular nic + promisc (with a perf penalty). > > > > SRIOV is not a requirement. And you can also use a dedicated > > nic+programmed mac if you are so inclined. > > Makes sense. Got it. > > I was going to add guest-to-guest to the test matrix, but I assume that > is not supported with vhost unless you have something like a VEPA > enabled bridge? > > <snip> Presumably you mean on the same host? There were also some patches to enable local guest to guest for macvlan, that would be a nice software-only solution. For back to back, I just tried over veth, seems to work fine. > >>> 3. add vhost=ethX > >> You mean via "ip link" I assume? > > > > No, that's a new flag for virtio in qemu: > > > > -net nic,model=virtio,vhost=veth0 > > Ah, ok. Even better. > > Thanks! > -Greg > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2009-08-12 16:39 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-08-11 21:27 [PATCHv2 0/2] vhost: a kernel-level virtio server Michael S. Tsirkin 2009-08-11 23:49 ` Gregory Haskins 2009-08-12 7:16 ` Michael S. Tsirkin 2009-08-12 11:56 ` Gregory Haskins 2009-08-12 12:05 ` Michael S. Tsirkin 2009-08-12 12:41 ` Gregory Haskins 2009-08-12 12:52 ` Arnd Bergmann 2009-08-12 13:06 ` Michael S. Tsirkin 2009-08-12 13:40 ` Arnd Bergmann 2009-08-12 13:42 ` Michael S. Tsirkin 2009-08-12 13:51 ` Gregory Haskins 2009-08-12 14:02 ` Michael S. Tsirkin 2009-08-12 16:13 ` Gregory Haskins 2009-08-12 16:37 ` Michael S. Tsirkin 2009-08-12 13:04 ` Michael S. Tsirkin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox