NetBSD Problem Report #42386

From liman@server.liman.net  Sun Nov 29 00:54:18 2009
Return-Path: <liman@server.liman.net>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 4389363C3B1
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 29 Nov 2009 00:54:18 +0000 (UTC)
Message-Id: <200911282342.nASNgHOj006207@server.liman.net>
Date: Sun, 29 Nov 2009 00:42:17 +0100 (MET)
From: liman@cafax.se
Reply-To: liman@cafax.se
To: gnats-bugs@gnats.NetBSD.org
Subject: amd64 dom0/domU IPv6 neighbor solicitations don't work.
X-Send-Pr-Version: 3.95

>Number:         42386
>Category:       port-xen
>Synopsis:       xen dom0 doesn't respond to IPv6 neighbor solicitations from domU
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    jdolecek
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Nov 29 00:55:00 +0000 2009
>Closed-Date:    Mon Jun 29 11:00:20 +0000 2020
>Last-Modified:  Mon Jun 29 11:00:20 +0000 2020
>Originator:     Lars-Johan Liman
>Release:        NetBSD 5.0.1
>Organization:
#-------------------------------------------------------------------------
# Lars-Johan Liman, M.Sc.                ! E-mail: liman@cafax.se
# Cafax AB                               ! HTTP  : //www.cafax.se/
# Computer Consultants, Sweden           ! Voice : +46 8 - 564 702 30
#-------------------------------------------------------------------------
>Environment:
System: NetBSD base.cafax.se 5.0.1 NetBSD 5.0.1 (XEN3_DOM0) #0: Thu Jul 30 00:23:51 UTC 2009  builds@b7.netbsd.org:/home/builds/ab/netbsd-5-0-1-RELEASE/amd64/200907292356Z-obj/home/builds/ab/netbsd-5-0-1-RELEASE/src/sys/arch/amd64/compile/XEN3_DOM0 amd64
Architecture: x86_64
Machine: amd64
>Description:

	[Maybe this should be in the networking category. I don't
	quite know ...]

	An amd64 dom0 does not recognize IPv6 neighbor solicitations
	(NS) from a domU that is connected via an internal bridge.

	The domU _does_ recognize the IPv6 neighbor solicitations
	coming from the dom0 over the same bridge.

	The NSs from the domU _do_ arrive at the dom0:tap0 (verified
	by tcpdump), but they do not trigger a neighbor advertisement
	(NA) from the dom0.

	When pinging from the dom0 towards the domU, and looking at
	the traffic at the domU:xennet0, tcpdump sees both an incoming
	NS, and an outgoing NA, which seems correct.

	The problem doesn't seem to pertain specifically to the tap0.
	If the domU:xennet0 is briged through to dom0:wm0 (or other
	external interface) instead of dom0:tap0, and the domU sends
	ping6 from xennet0 to a physical host on the wire, everything
	works fine. If the domU sends ping6 from xennet0 to dom0:wm0,
	it still doesn't work. Hence, I believe its the something in
	the processing of the NSs that doesn't work in the dom0, but
	it's odd that it _does_ work for packets coming in on the
	physical wm0 from other hosts, so it's the combination of dom0
	and domU somehow ... ?

	I have not investigated whether it pertains to other kinds of
	IPv6 anycast packets than NSs.

	It is also not related to the physical interface being a
	wm0. I've repeated the situation on a totally different
	machine which has a bge0 for external interface.

	IPv4 works just fine all the time.

	I humbly request assistance in solving the problem.

>How-To-Repeat:

	Prerequisites: amd64 machine, NetBSD 5.0.1, Xen 3.3,
		       NetBSD/amd64-dom0, NetBSD/amd64-domU

	dom0 configured as follows:
	     ip6mode: router
	     wm0: = external interface (can be other type than wm0, e.g. bge0)
	     	    inet6 2a00:A:B:C::2/64 alias (static config)
	     tap0: = internal interface towards domU
	     	    inet6 2a00:X:Y:Z::1/64 alias (static config)

	     bridge0: connects dom0:tap0 to domU:xennet0.

	domU configured as follows:
	     ip6mode: host
	     xennet0: = interface towards dom0
	     	    inet6 2a00:X:Y:Z::2/64 alias (static config)

	Now boot the machines and don't touch the IPv6 network (i.e.,
	don't send/receive any IPv6 packets).

	Login to domU (console or over IPv4) and try to ping6 the IPv6
	address on the dom0:tap0. No success. Leave the ping6 running
	in its non-working state.

	Login to dom0 (console or IPv4) and ping6 the IPv6 address of
	the domU:xennet0. Works just fine _AND_ as soon as the first
	ping has traversed the link, the non-working ping6 in the domU
	starts working, since the dom0 has now learnt the MAC address
	of the domU through its own NS process.

>Fix:

	None known. :-(

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: port-xen-maintainer->kern-bug-people
Responsible-Changed-By: bouyer@NetBSD.org
Responsible-Changed-When: Sun, 29 Nov 2009 16:03:28 +0000
Responsible-Changed-Why:
Not a Xen-specific problem, the issue is in the interration between bridge and
IPv6.


From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: tech-net@NetBSD.org, gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: kern/42386: amd64 dom0/domU IPv6 neighbor solicitations don't work.
Date: Sun, 29 Nov 2009 17:04:05 +0100

 On Sun, Nov 29, 2009 at 12:55:00AM +0000, liman@cafax.se wrote:
 > 	An amd64 dom0 does not recognize IPv6 neighbor solicitations
 > 	(NS) from a domU that is connected via an internal bridge.
 > 
 > 	The domU _does_ recognize the IPv6 neighbor solicitations
 > 	coming from the dom0 over the same bridge.
 > 
 > 	The NSs from the domU _do_ arrive at the dom0:tap0 (verified
 > 	by tcpdump), but they do not trigger a neighbor advertisement
 > 	(NA) from the dom0.
 > 
 > 	When pinging from the dom0 towards the domU, and looking at
 > 	the traffic at the domU:xennet0, tcpdump sees both an incoming
 > 	NS, and an outgoing NA, which seems correct.
 > 
 > 	The problem doesn't seem to pertain specifically to the tap0.
 > 	If the domU:xennet0 is briged through to dom0:wm0 (or other
 > 	external interface) instead of dom0:tap0, and the domU sends
 > 	ping6 from xennet0 to a physical host on the wire, everything
 > 	works fine. If the domU sends ping6 from xennet0 to dom0:wm0,
 > 	it still doesn't work. Hence, I believe its the something in
 > 	the processing of the NSs that doesn't work in the dom0, but
 > 	it's odd that it _does_ work for packets coming in on the
 > 	physical wm0 from other hosts, so it's the combination of dom0
 > 	and domU somehow ... ?
 > 
 > 	I have not investigated whether it pertains to other kinds of
 > 	IPv6 anycast packets than NSs.
 > 
 > 	It is also not related to the physical interface being a
 > 	wm0. I've repeated the situation on a totally different
 > 	machine which has a bge0 for external interface.

 I think this is an issue because the packets are bridged.
 From what I've seen the incoming neighbor solicitation on dom0
 is dropped and accounts in "multicast packets which we don't join"
 in netstat -pinet6 output (and also "packets not forwardable").
 It's being dropped in this code part of ip6_input():
         if (IN6_IS_ADDR_MULTICAST(&ip6->ip6_dst)) {
                 struct  in6_multi *in6m = 0;

                 in6_ifstat_inc(m->m_pkthdr.rcvif, ifs6_in_mcast);
                 /*
                  * See if we belong to the destination multicast group on the
                  * arrival interface.
                  */
                 IN6_LOOKUP_MULTI(ip6->ip6_dst, m->m_pkthdr.rcvif, in6m);
                 if (in6m)
                         ours = 1;
                 else if (!ip6_mrouter) {
                         uint64_t *ip6s = IP6_STAT_GETREF();
                         ip6s[IP6_STAT_NOTMEMBER]++;
                         ip6s[IP6_STAT_CANTFORWARD]++;
                         IP6_STAT_PUTREF();
                         in6_ifstat_inc(m->m_pkthdr.rcvif, ifs6_in_discard);
                         goto bad;
                 }
                 deliverifp = m->m_pkthdr.rcvif;
                 goto hbhcheck;
         }

 on my system RAs from domU to dom0 are sent to ff02::1:ff00:1, ether
 multicast 33:33:ff:00:00:01 while RAs from dom0 to domU are sent
 to ff02::1:ff00:2, ether mutlticast 33:33:ff:00:00:02

 I don't know why the domU responds to ff02::1:ff00:2 while the dom0
 doesn't reply to ff02::1:ff00:1

 dom0 replies to a ping6 ff02::1:ff00:1%wm0 from dom0,  but not the domU.
 domU replies to a ping6 ff02::1:ff00:2%xvif1.0 from dom0, but not the dom0.
 Nothing replies to ping6 ff02::1:ff00:2%wm0

 from the domU, no replies from ping6 ff02::1:ff00:1%xennet0,
 domU replies to ff02::1:ff00:2%xennet0

 I don't know what the correct behavior is supposed to be, nor how to
 fix the problem (exept keeping a ping6 in the background on dom0 to
 make sure the domUs have the right entries in their routing
 table).

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

State-Changed-From-To: open->feedback
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Tue, 19 Jun 2018 21:31:57 +0000
State-Changed-Why:
Does this still happen?


From: Lars-Johan Liman <liman@cafax.se>
To: jdolecek@NetBSD.org
Cc: kern-bug-people@netbsd.org, netbsd-bugs@netbsd.org, gnats-admin@netbsd.org,
        gnats-bugs@NetBSD.org
Subject: Re: port-xen/42386 (xen dom0 doesn't respond to IPv6 neighbor solicitations from domU)
Date: Wed, 20 Jun 2018 10:17:12 +0200

 jdolecek@NetBSD.org:
 > Synopsis: xen dom0 doesn't respond to IPv6 neighbor solicitations from domU

 > State-Changed-From-To: open->feedback
 > State-Changed-By: jdolecek@NetBSD.org
 > State-Changed-When: Tue, 19 Jun 2018 21:31:57 +0000
 > State-Changed-Why:
 > Does this still happen?

 It still happens with 7.1.2 on both dom0 and domU (precompiled installed
 from ISO). Just tested.

 I don't have a system with an 8.* dom0 that I can test. :-(

 				Best regards,
 				  /Liman
 #-------------------------------------------------------------------------
 # Lars-Johan Liman, M.Sc.		 ! E-mail: liman@cafax.se
 # Cafax AB				 ! HTTP  : //www.cafax.se/
 # Computer Consultants, Sweden		 ! Voice : +46 8 - 564 702 30
 #-------------------------------------------------------------------------

From: Lars-Johan Liman <liman@cafax.se>
To: kern-bug-people@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, gnats-bugs@NetBSD.org
Subject: Re: port-xen/42386 (xen dom0 doesn't respond to IPv6 neighbor solicitations from domU)
Date: Wed, 20 Jun 2018 10:53:45 +0200

 jdolecek@NetBSD.org:
 >> Does this still happen?

 liman@cafax.se:
 > It still happens with 7.1.2 on both dom0 and domU (precompiled installed
 > from ISO). Just tested.

 I realise that was an ambigous statement.

 New attempt:

 The problem described in the intital report still happens even when both
 the domU and the dom0 are running 7.1.2 (= latest stable version).

 				Cheers,
 				  /Liman

State-Changed-From-To: feedback->open
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Wed, 20 Jun 2018 11:05:21 +0000
State-Changed-Why:
Confirmed to still happen with newer versions. Thanks.


Responsible-Changed-From-To: kern-bug-people->jdolecek
Responsible-Changed-By: jdolecek@NetBSD.org
Responsible-Changed-When: Mon, 23 Mar 2020 17:58:49 +0000
Responsible-Changed-Why:
I've encountered this bug while testing IPv6 offloading for xennet(4),
let's see if I can figure out a fix.


From: "Jaromir Dolecek" <jdolecek@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42386 CVS commit: src/sys/net
Date: Tue, 24 Mar 2020 13:30:55 +0000

 Module Name:	src
 Committed By:	jdolecek
 Date:		Tue Mar 24 13:30:54 UTC 2020

 Modified Files:
 	src/sys/net: if_bridge.c

 Log Message:
 reset the csum_flags in bridge_brodcast() also for bmcast path

 for destination interfaces with real hardware offloading this fixes
 multicast packet corruption; for xvif(4) this fix stops treating them
 as having no csum

 may fix PR kern/42386


 To generate a diff of this commit:
 cvs rdiff -u -r1.168 -r1.169 src/sys/net/if_bridge.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Tue, 24 Mar 2020 13:50:35 +0000
State-Changed-Why:
Can you please check kernel with rev. 1.169 of sys/net/if_bridge.c? With
that revision I can no longer repeat the problem on my setup, which
is admittably somewhat different.


From: Lars-Johan Liman <liman@cafax.se>
To: jdolecek@NetBSD.org
Cc: netbsd-bugs@netbsd.org, gnats-admin@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: port-xen/42386 (xen dom0 doesn't respond to IPv6 neighbor solicitations from domU)
Date: Wed, 25 Mar 2020 11:24:37 +0100

 jdolecek@NetBSD.org 2020-03-24 13:50 [+0000]:
 > Synopsis: xen dom0 doesn't respond to IPv6 neighbor solicitations from domU

 > State-Changed-From-To: open->feedback
 > State-Changed-By: jdolecek@NetBSD.org
 > State-Changed-When: Tue, 24 Mar 2020 13:50:35 +0000
 > State-Changed-Why:
 > Can you please check kernel with rev. 1.169 of sys/net/if_bridge.c? With
 > that revision I can no longer repeat the problem on my setup, which
 > is admittably somewhat different.

 Hi!

 I'll see what I can do. I may need a day or two to make this happen,
 though. I don't have a lot of lab h/w to play with. Nor time ... :-(

 				Best regards,
 				  /Liman
 -- 
 #-------------------------------------------------------------------------
 # Lars-Johan Liman, M.Sc.		 ! HTTP  : //www.cafax.se/
 # Cafax AB				 !
 # Computer Consultants, Sweden		 !
 #-------------------------------------------------------------------------

State-Changed-From-To: feedback->closed
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Mon, 29 Jun 2020 11:00:20 +0000
State-Changed-Why:
Hopefully fixed in -current. If you happen to get around to test
it on your system and it still doesn't work, please mail me to reopen the PR.
Thanks for report and somewhat delayed response.


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.