NetBSD Problem Report #50290

From www@NetBSD.org  Tue Sep 29 09:50:48 2015
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 2075DA654F
	for <gnats-bugs@gnats.NetBSD.org>; Tue, 29 Sep 2015 09:50:48 +0000 (UTC)
Message-Id: <20150929095046.B9275A6554@mollari.NetBSD.org>
Date: Tue, 29 Sep 2015 09:50:46 +0000 (UTC)
From: netbsd@precedence.co.uk
Reply-To: netbsd@precedence.co.uk
To: gnats-bugs@NetBSD.org
Subject: Regression: xennet very slow in netbsd-7
X-Send-Pr-Version: www-1.0

>Number:         50290
>Category:       port-xen
>Synopsis:       Regression: xennet very slow in netbsd-7
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    port-xen-maintainer
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Sep 29 09:55:00 +0000 2015
>Closed-Date:    Sat Mar 14 12:03:01 +0000 2020
>Last-Modified:  Fri Apr 24 13:55:01 +0000 2020
>Originator:     Stephen Borrill
>Release:        7.0_RC3
>Organization:
>Environment:
NetBSD nmdevel7 7.0_RC3 NetBSD 7.0_RC3 (NETMANXEN3) #24: Mon Sep  7 11:38:06 BST 2015  root@builder7:/usr/work/netmanager/work/obj/7.0/sys/arch/amd64/compile/NETMANXEN3 amd64
>Description:
Network throughput with the xennet driver on netbsd-7 is 10% of netbsd-5. Testbed is physical netbsd-5 host as netio client, with netio servers runs on netbsd-5 and netbsd-7 Xen domUs on the same XenServer host.

NetBSD 5.2 almost saturates the 1Gb link:
TCP connection established.
Packet size  1k bytes:  103375 KByte/s Tx,  81319 KByte/s Rx.
Packet size  2k bytes:  107507 KByte/s Tx,  96727 KByte/s Rx.
Packet size  4k bytes:  110407 KByte/s Tx,  94854 KByte/s Rx.
Packet size  8k bytes:  111326 KByte/s Tx,  108494 KByte/s Rx.
Packet size 16k bytes:  111244 KByte/s Tx,  107681 KByte/s Rx.
Packet size 32k bytes:  111291 KByte/s Tx,  99243 KByte/s Rx.

NetBSD 7.0_RC3 has 10% of throughput:
Packet size  1k bytes:  9641 KByte/s Tx,  9890 KByte/s Rx.
Packet size  2k bytes:  11940 KByte/s Tx,  11718 KByte/s Rx.
Packet size  4k bytes:  11529 KByte/s Tx,  12110 KByte/s Rx.
Packet size  8k bytes:  11485 KByte/s Tx,  12266 KByte/s Rx.
Packet size 16k bytes:  11435 KByte/s Tx,  12051 KByte/s Rx.
Packet size 32k bytes:  11501 KByte/s Tx,  12287 KByte/s Rx.

>How-To-Repeat:
As above
>Fix:

>Release-Note:

>Audit-Trail:
From: Stephen Borrill <netbsd@precedence.co.uk>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-xen/50290: Regression: xennet very slow in netbsd-7
Date: Tue, 29 Sep 2015 12:51:44 +0100 (BST)

 This is down to options MBUFTRACE. On netbsd-5 this has little or no 
 impact. On netbsd-7, it kills performance. Is this a bug or expected?

From: Stephen Borrill <netbsd@precedence.co.uk>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-xen/50290: Regression: xennet very slow in netbsd-7
Date: Wed, 30 Sep 2015 09:52:40 +0100 (BST)

 On Tue, 29 Sep 2015, Stephen Borrill wrote:

 > This is down to options MBUFTRACE. On netbsd-5 this has little or no impact. 
 > On netbsd-7, it kills performance. Is this a bug or expected?

 I've tested on physical hardware with kernels from the same 
 versions of the source tree. MBUFTRACE has no effect with non-xennet 
 interfaces:

 Packet size  1k bytes:  100650 KByte/s Tx,  91729 KByte/s Rx.
 Packet size  2k bytes:  107367 KByte/s Tx,  106770 KByte/s Rx.
 Packet size  4k bytes:  109194 KByte/s Tx,  109567 KByte/s Rx.
 Packet size  8k bytes:  104674 KByte/s Tx,  108502 KByte/s Rx.
 Packet size 16k bytes:  107622 KByte/s Tx,  110378 KByte/s Rx.
 Packet size 32k bytes:  108061 KByte/s Tx,  111504 KByte/s Rx.

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: port-xen-maintainer@netbsd.org, gnats-admin@netbsd.org,
        netbsd-bugs@netbsd.org, netbsd@precedence.co.uk
Subject: Re: port-xen/50290: Regression: xennet very slow in netbsd-7
Date: Mon, 5 Oct 2015 17:10:37 +0200

 On Tue, Sep 29, 2015 at 11:55:00AM +0000, Stephen Borrill wrote:
 > The following reply was made to PR port-xen/50290; it has been noted by GNATS.
 > 
 > From: Stephen Borrill <netbsd@precedence.co.uk>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: port-xen/50290: Regression: xennet very slow in netbsd-7
 > Date: Tue, 29 Sep 2015 12:51:44 +0100 (BST)
 > 
 >  This is down to options MBUFTRACE. On netbsd-5 this has little or no 
 >  impact. On netbsd-7, it kills performance. Is this a bug or expected?

 The xnenet mbuf code didn't change much between netbsd-5 and -7; the MCLAIM
 calls are the same. Did you try a bare-metal kernel with MBUFTRACE ?

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

State-Changed-From-To: open->closed
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Sat, 14 Mar 2020 12:03:01 +0000
State-Changed-Why:
This is due to MBUFTRACE option, just don't use it.


From: "Jaromir Dolecek" <jdolecek@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/50290 CVS commit: src/share/man/man4
Date: Fri, 24 Apr 2020 13:47:50 +0000

 Module Name:	src
 Committed By:	jdolecek
 Date:		Fri Apr 24 13:47:50 UTC 2020

 Modified Files:
 	src/share/man/man4: options.4

 Log Message:
 actually MBUFTRACE does splvm(), kpreempt_disable()/enable() and percpu,
 this has non-slight overhead - amend documentation to stop claiming
 the overhead is slight

 adresses PR port-xen/50290


 To generate a diff of this commit:
 cvs rdiff -u -r1.510 -r1.511 src/share/man/man4/options.4

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: =?UTF-8?B?SmFyb23DrXIgRG9sZcSNZWs=?= <jaromir.dolecek@gmail.com>
To: "gnats-bugs@NetBSD.org" <gnats-bugs@netbsd.org>
Cc: 
Subject: Re: port-xen/50290 (Regression: xennet very slow in netbsd-7)
Date: Fri, 24 Apr 2020 15:50:17 +0200

 FYI - contrary to xennet(4) most network adapters don't call MCLAIM()
 so don't actually support MBUFTRACE and are not hit by eventual
 performance problems.

 mowner_claim(), invoked via MCLAIM()->m_claim() manipulates SPL,
 disables/enables preemption via percpu_getref(), and I believe also
 does IPIs. All of these are relatively expensive on XenPV, that's why
 it's somewhat disproportionally hit.

 I've updated the documentation in options(4) to make it reflect more
 accurately that it does, in fact, affect the network performance
 significantly.

>Unformatted:

Home	PR Database Search