NetBSD Problem Report #51531

From gson@gson.org  Wed Oct  5 13:59:52 2016
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 375B87A281
	for <gnats-bugs@gnats.NetBSD.org>; Wed,  5 Oct 2016 13:59:52 +0000 (UTC)
Message-Id: <20161005135943.0FDE37449CD@guava.gson.org>
Date: Wed,  5 Oct 2016 16:59:43 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: Recent networking regression affecting installs
X-Send-Pr-Version: 3.95

>Number:         51531
>Category:       kern
>Synopsis:       Recent networking regression affecting installs
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 05 14:00:00 +0000 2016
>Last-Modified:  Thu Dec 15 17:50:00 +0000 2016
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current, source date >= 2016.09.16.23.01.53
>Organization:
>Environment:
System: NetBSD
Architecture: i386
Machine: i386
>Description:

I have an automated testing setup that netboots an INSTALL kernel and
performs a scripted installation of NetBSD/i386 or /amd64 on physical
hardware using sysinst.

This stopped working on both i386 and amd64 some time during the
period of build breakage lasting from 2016.09.16.11.35.07 to
2016.09.16.23.01.53, and is still broken as of 2016.10.05.03.46.38.

The ping test performed by sysinst fails with "100.0% packet loss, and
then the HTTP download of the first installation set fails:

  ftp: Can't connect to `10.169.0.1:80': Operation timed out

A log from the most recent installation attempt is here, but a bit
hard to read because it contains terminal escape sequences from
sysinst (with the ESC characters stripped):

  http://www.gson.org/netbsd/bugs/build/i386-baremetal/2016/2016.10.05.03.46.38/install.log

The problem started with one of the following commits:

   2016.09.16.11.35.07 jdolecek src/sys/dev/pci/nvme_pci.c 1.6
   2016.09.16.11.35.07 jdolecek src/sys/modules/Makefile 1.177
   2016.09.16.11.35.07 jdolecek src/sys/modules/nvme/Makefile 1.1
   2016.09.16.11.35.07 jdolecek src/sys/modules/nvme/nvme.ioconf 1.1
   2016.09.16.11.41.40 jdolecek src/sys/dev/ic/nvme.c 1.6
   2016.09.16.11.48.10 maxv src/sys/arch/amd64/amd64/trap.c 1.85
   2016.09.16.11.48.10 maxv src/sys/arch/i386/i386/trap.c 1.279
   2016.09.16.12.28.41 maxv src/sys/arch/i386/i386/copy.S 1.25
   2016.09.16.12.31.27 jdolecek src/share/man/man4/nvme.4 1.3
   2016.09.16.12.43.37 wiz src/share/man/man4/nvme.4 1.4
   2016.09.16.12.57.26 jdolecek src/sys/dev/ic/nvme.c 1.7
   2016.09.16.13.47.47 roy src/sys/netinet/if_arp.c 1.226
   2016.09.16.13.56.36 jdolecek src/share/man/man9/pci_msi.9 1.13
   2016.09.16.14.17.23 roy src/sys/net/if_spppsubr.c 1.152
   2016.09.16.14.17.23 roy src/sys/netinet/in.c 1.182
   2016.09.16.14.17.23 roy src/sys/netinet/in_var.h 1.83
   2016.09.16.14.55.28 jdolecek src/doc/roadmaps/storage 1.16
   2016.09.16.15.02.23 jdolecek src/doc/roadmaps/storage 1.17
   2016.09.16.15.20.50 jdolecek src/sys/dev/ata/ld_ataraid.c 1.42
   2016.09.16.15.20.50 jdolecek src/sys/dev/i2o/ld_iop.c 1.36
   2016.09.16.15.20.50 jdolecek src/sys/dev/ic/ld_aac.c 1.29
   2016.09.16.15.20.50 jdolecek src/sys/dev/ic/ld_cac.c 1.29
   2016.09.16.15.20.50 jdolecek src/sys/dev/ic/ld_icp.c 1.28
   2016.09.16.15.20.50 jdolecek src/sys/dev/ic/ld_mlx.c 1.22
   2016.09.16.15.20.50 jdolecek src/sys/dev/ic/ld_nvme.c 1.2
   2016.09.16.15.20.50 jdolecek src/sys/dev/ld.c 1.95
   2016.09.16.15.20.50 jdolecek src/sys/dev/ldvar.h 1.28
   2016.09.16.15.20.50 jdolecek src/sys/dev/pci/ld_amr.c 1.24
   2016.09.16.15.20.50 jdolecek src/sys/dev/pci/ld_twa.c 1.18
   2016.09.16.15.20.50 jdolecek src/sys/dev/pci/ld_twe.c 1.38
   2016.09.16.15.20.50 jdolecek src/sys/dev/pci/ld_virtio.c 1.11
   2016.09.16.15.20.50 jdolecek src/sys/dev/sdmmc/ld_sdmmc.c 1.22
   2016.09.16.15.24.47 jdolecek src/sys/dev/ic/ld_nvme.c 1.3
   2016.09.16.17.12.06 christos src/lib/libc/time/Makefile 1.33
   2016.09.16.17.12.06 christos src/lib/libc/time/NEWS 1.16
   2016.09.16.17.12.06 christos src/lib/libc/time/Theory 1.20
   2016.09.16.17.12.06 christos src/lib/libc/time/tz-art.htm 1.13
   2016.09.16.17.12.06 christos src/lib/libc/time/tz-how-to.html 1.1
   2016.09.16.17.12.06 christos src/lib/libc/time/tz-link.htm 1.26
   2016.09.16.17.12.06 christos src/lib/libc/time/zic.c 1.59
   2016.09.16.17.13.06 christos src/doc/3RDPARTY 1.1361
   2016.09.16.17.13.06 christos src/doc/CHANGES 1.2185
   2016.09.16.17.27.09 matt src/sys/uvm/pmap/pmap.c 1.22
   2016.09.16.17.32.36 scole src/sys/arch/macppc/dev/platinumfb.c 1.3
   2016.09.16.20.30.14 christos src/external/gpl3/gcc/dist/gcc/config/m68k/m68k.md 1.5
   2016.09.16.20.31.00 christos src/external/gpl3/gcc/dist/gcc/config/m68k/m68k.c 1.2
   2016.09.16.22.39.35 macallan src/sys/arch/sparc/dev/cgfourteen.c 1.82
   2016.09.16.23.01.53 pgoyette src/distrib/sets/lists/modules/mi 1.95

Those by roy are the only ones that look networking related to me.

>How-To-Repeat:

Attempt a network install of -current.

>Fix:

>Release-Note:

>Audit-Trail:
From: Roy Marples <roy@NetBSD.org>
To: Andreas Gustafsson <gson@gson.org>, gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 5 Oct 2016 15:11:53 +0100

 Hi Andreas

 On 05/10/2016 15:03, Andreas Gustafsson wrote:
 > Could you please have a look at PR kern/51531, since it looks like you
 > were the only one to touch the networking code during the period when
 > the problem appeared?

 I think ifconfig -w 15 -W 5 should be added to sysinst before the ping
 test to wait for DaD to clear.

 Roy

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: kern-bug-people@netbsd.org,
    gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org,
    gson@gson.org (Andreas Gustafsson)
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 5 Oct 2016 17:42:18 +0300

 Roy Marples wrote:
 >  I think ifconfig -w 15 -W 5 should be added to sysinst before the ping
 >  test to wait for DaD to clear.

 I think sysinst, and other existing code that configures IP addresses,
 is not incorrect as written, and should not be required to do this.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@NetBSD.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org, Andreas Gustafsson <gson@gson.org>
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 5 Oct 2016 17:07:34 +0200

 On Wed, Oct 05, 2016 at 02:45:01PM +0000, Andreas Gustafsson wrote:
 > The following reply was made to PR kern/51531; it has been noted by GNATS.
 > 
 > From: Andreas Gustafsson <gson@gson.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: kern-bug-people@netbsd.org,
 >     gnats-admin@netbsd.org,
 >     netbsd-bugs@netbsd.org,
 >     gson@gson.org (Andreas Gustafsson)
 > Subject: Re: kern/51531: Recent networking regression affecting installs
 > Date: Wed, 5 Oct 2016 17:42:18 +0300
 > 
 >  Roy Marples wrote:
 >  >  I think ifconfig -w 15 -W 5 should be added to sysinst before the ping
 >  >  test to wait for DaD to clear.
 >  
 >  I think sysinst, and other existing code that configures IP addresses,
 >  is not incorrect as written, and should not be required to do this.

 I disagree, quite a bit. There are three options for such a program:
 (1) It is dumb and just expects a newly configured interface to work
 after some time. Nothing new here.
 (2) It tries to be smart and checks immediately after configuration if
 ping works. This has been wrong for a variety of network conditions
 already, since link configuration itself can already take time in the
 second range. WiFi can be even worse with slow authorisation. This
 really is an application bug and not related to this issue at all.
 (3) It tries to be even smarter and checks for link state changes before
 ping. This still doesn't work properly for WiFi or 802.1x ports. DaD
 timeouts are no different here.

 In short, *any* assumption that a freshly configured network interface
 is going to be ready RSN are wrong. There are some architectural issues
 in that we currently have no way to report "this interface should be
 usable now", which is shared with all the BSDs. But it is essentially
 unrelated to the recent DaD changes.

 Joerg

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 5 Oct 2016 20:20:36 +0000 (UTC)

 joerg@bec.de (Joerg Sonnenberger) writes:

 >In short, *any* assumption that a freshly configured network interface
 >is going to be ready RSN are wrong.

 Of course nobody really cared because in practice it just worked.

 It started to be wrong when enforcing DAD.
 It is so well-known to be wrong that people invented optimistic DAD
 to reduce the extra delay, but so far only for IPv6. DAD on IPv4
 seems to be less common.

 -- 
 -- 
                                 Michael van Elst
 Internet: mlelstv@serpens.de
                                 "A potential Snark may lurk in every tree."

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@NetBSD.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Mon, 21 Nov 2016 18:50:42 +0200

 Roy,

 On October 5, you wrote:
 > I think ifconfig -w 15 -W 5 should be added to sysinst before the ping
 > test to wait for DaD to clear.

 And I responded:
 < I think sysinst, and other existing code that configures IP addresses,
 < is not incorrect as written, and should not be required to do this.

 Please don't let our disagreement over the correct way to fix this
 issue keep you from fixing it.  If you won't fix it my way, then fix
 it your way.
 -- 
 Andreas Gustafsson, gson@gson.org

From: "Roy Marples" <roy@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Tue, 22 Nov 2016 12:04:35 +0000

 Module Name:	src
 Committed By:	roy
 Date:		Tue Nov 22 12:04:35 UTC 2016

 Modified Files:
 	src/usr.sbin/sysinst: net.c

 Log Message:
 Fix PR kern/51531 by using ifconfig to wait for addresses to become
 valid rather than sleeping a fixed ammount of time.


 To generate a diff of this commit:
 cvs rdiff -u -r1.21 -r1.22 src/usr.sbin/sysinst/net.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Roy Marples <roy@NetBSD.org>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Tue, 22 Nov 2016 12:05:24 +0000

 Hi

 On 21/11/2016 16:50, Andreas Gustafsson wrote:
 > On October 5, you wrote:
 >> I think ifconfig -w 15 -W 5 should be added to sysinst before the ping
 >> test to wait for DaD to clear.
 > 
 > And I responded:
 > < I think sysinst, and other existing code that configures IP addresses,
 > < is not incorrect as written, and should not be required to do this.
 > 
 > Please don't let our disagreement over the correct way to fix this
 > issue keep you from fixing it.  If you won't fix it my way, then fix
 > it your way.

 Fixed.

 I don't have the resource to actually test it though.

 Roy

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@NetBSD.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Tue, 22 Nov 2016 20:03:37 +0200

 Roy Marples wrote:
 > Fixed.
 > 
 > I don't have the resource to actually test it though.

 I ran an automated test on -current sources from after the commit of
 sysinst/net.c 1.22, and the install still failed.  The install log is
 here:

   http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.11.22.12.04.35/install.log

 Based on the log:

  - The command "/sbin/ifconfig -w 15 -W 5" is now being run
  - The ping test fails with 100% packet loss as before
  - Retrieving the first installation set fails as before, with
    "ftp: Can't connect to `10.169.01:80': Operation timed out"
  - Running ifconfig after the failed install shows the
    TENTATIVE flag is set:

     sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
 	 ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
 	 ec_enabled=0
 	 address: 00:1b:fc:9e:0f:b4
 	 media: Ethernet autoselect (10baseT half-duplex)
 	 status: active
 	 inet 10.169.0.2/24 broadcast 10.169.0.255 flags 0x1<TENTATIVE>
 	 inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x2<TENTATIVE> scopeid 0x3

 -- 
 Andreas Gustafsson, gson@gson.org

From: Roy Marples <roy@marples.name>
To: Andreas Gustafsson <gson@gson.org>,Roy Marples <roy@NetBSD.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Tue, 22 Nov 2016 18:40:29 +0000

 ------R3199Y3SOPHUWPG8W7NG3I01JVO8QX
 Content-Transfer-Encoding: 8bit
 Content-Type: text/plain;
  charset=UTF-8

 Do you know if the new ifconfig command did in fact wait for around 15 seconds before running ping?

 On 22 November 2016 18:03:37 GMT+00:00, Andreas Gustafsson <gson@gson.org> wrote:
 >Roy Marples wrote:
 >> Fixed.
 >> 
 >> I don't have the resource to actually test it though.
 >
 >I ran an automated test on -current sources from after the commit of
 >sysinst/net.c 1.22, and the install still failed.  The install log is
 >here:
 >
 >http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.11.22.12.04.35/install.log
 >
 >Based on the log:
 >
 > - The command "/sbin/ifconfig -w 15 -W 5" is now being run
 > - The ping test fails with 100% packet loss as before
 > - Retrieving the first installation set fails as before, with
 >   "ftp: Can't connect to `10.169.01:80': Operation timed out"
 > - Running ifconfig after the failed install shows the
 >   TENTATIVE flag is set:
 >
 >    sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
 >	 ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
 >	 ec_enabled=0
 >	 address: 00:1b:fc:9e:0f:b4
 >	 media: Ethernet autoselect (10baseT half-duplex)
 >	 status: active
 >	 inet 10.169.0.2/24 broadcast 10.169.0.255 flags 0x1<TENTATIVE>
 >	 inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x2<TENTATIVE> scopeid 0x3
 >
 >-- 
 >Andreas Gustafsson, gson@gson.org

 -- 
 Sent from my Android device with K-9 Mail. Please excuse my brevity.
 ------R3199Y3SOPHUWPG8W7NG3I01JVO8QX
 Content-Type: text/html;
  charset=utf-8
 Content-Transfer-Encoding: 8bit

 <html><head></head><body>Do you know if the new ifconfig command did in fact wait for around 15 seconds before running ping?<br><br><div class="gmail_quote">On 22 November 2016 18:03:37 GMT+00:00, Andreas Gustafsson &lt;gson@gson.org&gt; wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
 <pre class="k9mail">Roy Marples wrote:<br /><blockquote class="gmail_quote" style="margin: 0pt 0pt 1ex 0.8ex; border-left: 1px solid #729fcf; padding-left: 1ex;"> Fixed.<br /> <br /> I don't have the resource to actually test it though.<br /></blockquote><br />I ran an automated test on -current sources from after the commit of<br />sysinst/net.c 1.22, and the install still failed.  The install log is<br />here:<br /><br />  <a href="http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.11.22.12.04.35/install.log">http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.11.22.12.04.35/install.log</a><br /><br />Based on the log:<br /><br /> - The command "/sbin/ifconfig -w 15 -W 5" is now being run<br /> - The ping test fails with 100% packet loss as before<br /> - Retrieving the first installation set fails as before, with<br />   "ftp: Can't connect to `10.169.01:80': Operation timed out"<br /> - Running ifconfig after the failed install shows the<br />  
 TENTATIVE flag is set:<br /><br />    sk0: flags=0x8843&lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST&gt; mtu 1500<br />  ec_capabilities=5&lt;VLAN_MTU,JUMBO_MTU&gt;<br />  ec_enabled=0<br />  address: 00:1b:fc:9e:0f:b4<br />  media: Ethernet autoselect (10baseT half-duplex)<br />  status: active<br />  inet <a href="http://10.169.0.2/24">10.169.0.2/24</a> broadcast <a href="http://10.169.0.255">10.169.0.255</a> flags 0x1&lt;TENTATIVE&gt;<br />  inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x2&lt;TENTATIVE&gt; scopeid 0x3<br /></pre></blockquote></div><br>
 -- <br>
 Sent from my Android device with K-9 Mail. Please excuse my brevity.</body></html>
 ------R3199Y3SOPHUWPG8W7NG3I01JVO8QX--

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Tue, 22 Nov 2016 21:05:05 +0200

 Roy Marples wrote:
 > Do you know if the new ifconfig command did in fact wait for around
 > 15 seconds before running ping?

 I don't know, as I do not have a timestamped log.  I will add a
 timestamping facility and rerun the test tomorrow.

 I should also mention that when the machine netboots, it gets the
 address 10.169.0.2 using DHCP at the PXE and tftpboot stages, and
 then in sysinst, it is statically configured with the same address it
 previously got using DHCP.  Could this be confusing DAD somehow?

 This is a dedicated network segment with only the server (10.169.0.1)
 and the machine being installed on, so there should be no possibility
 of address conflicts with third parties.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Roy Marples <roy@marples.name>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Tue, 22 Nov 2016 19:58:40 +0000

 On 22/11/2016 19:05, Andreas Gustafsson wrote:
 > Roy Marples wrote:
 >> Do you know if the new ifconfig command did in fact wait for around
 >> 15 seconds before running ping?
 > 
 > I don't know, as I do not have a timestamped log.  I will add a
 > timestamping facility and rerun the test tomorrow.
 > 
 > I should also mention that when the machine netboots, it gets the
 > address 10.169.0.2 using DHCP at the PXE and tftpboot stages, and
 > then in sysinst, it is statically configured with the same address it
 > previously got using DHCP.  Could this be confusing DAD somehow?
 > 
 > This is a dedicated network segment with only the server (10.169.0.1)
 > and the machine being installed on, so there should be no possibility
 > of address conflicts with third parties.

 I'm going to guess that ifconfig -w 15 -W 5 did infact wait the time.
 I'll also guess that my recentish changes have unmasked the actual
 problem in that for some reason DaD isn't completing on your interface.

 Could you wait a few minutes and then look at the ifconfig output again?
 Guessing the addresses still say TENTATIVE.

 Roy

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 07:16:23 +0700

 In all of this (and regardless of whatever bugs may be uncovered in
 IPv4 DaD processing, and which clearly should get fixed), I still
 fail to understand this fetish of not sending from a tentative address.

 Note, tentative is not invalid - it is an address whose address is not
 invalid but that is not yet known for certain.

 I haven't looked at whatever spec now exists for testing v4 addresses
 in this way, but as I recall it, for v6, it is absolutely required to
 send from a tentative address in some circumstances (particularly when
 2 hosts are attempting to configure the same address at the same time,
 and both are performing DaD ... if both stay silent, neither will see
 the conflict, and both will assume they can transition the addr to valid.)

 The real fix for this problem (in addition to any others that come to
 attention because of it) would seem to be to remove the restriction on
 sending from tentative addresses (that is, to stop treating them as invalid.)

 Almost always, DaD "succeeds" and the tentative address becomes valid.
 This may not have been quite so true in the past, when people commonly
 manually configured IPv4 addresses, and it what let to using ARP to check,
 which eventually led to DaD for IPv6, and then its formalisation for IPv4.
 These days, almost no-one manually configures addresses, and when they do
 it is most commonly of routers/major servers, and they're very rarely
 configured incorrectly, and automatic addr assignment (DHCP, or for v6,
 autoconfig) almost never produces duplicates.

 But - for DaD to succeed, we must wait - if we really believe that there's
 more than a negligible chance of duplicates, we really should be sending
 multiple queries, and waiting after each.   That means noticeable delays,
 which almost always are pointless.

 On the other hand when DaD does happen to fail (the addr is duplicated)
 it almost always fails very very quickly (usually within a millisecond.)
 That is, the other host to which the addr has been assigned responds to
 the DaD query very very quickly.   That is, usually, by the time we have
 finished configuring the address, and are ready to actually send a packet
 if DaD were ever going to fail, it almost certainly always will have.
 And if we want to reduce our chances of accidentally using addr that might
 become invalid very soon, we can just delay, 10 ms or so should be enough,
 after configuring the address.   If DaD has not failed by then, the chances
 are very small that it ever will - and if we wanted to improve the chances,
 we could send the second query about 5ms after the first rather than waiting
 for the entire timeout - just in case something happened to the first packet
 (corrupted on the wire, or similar) so it had no chance to be received.

 Also note that there is 0 chance of never using an address that might
 become invalid in the future (and no way of knowing how far in the future
 that might happen) so we either need to simply tolerate occasional comms
 failures (which is what we mostly just do - just try again if things don't
 work first time - for any reason) or apps need to get a lot smarter to
 learn to recover from such events.   That is, avoiding using a tentative
 address, because it might become invalid is silly, because any address
 might become invalid.   Even avoiding it because it might become duplicated
 is pointless, as the (admittedly rare) case of 2 net segments, reconnecting
 after the same address happened to be assigned (and DaD carried out without
 complaint) on each of the segments before they re-joined shows.

 Please, just put the networking code back to the state it used to be, and
 allow tentative addresses to be considered valid for the purposes of
 packet sending/reception.

 kre

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 10:33:02 +0200

 Roy Marples wrote:
 > >> Do you know if the new ifconfig command did in fact wait for around
 > >> 15 seconds before running ping?

 It waits for about 5 seconds, not 15.  Here's the relevant part
 of the timestamped log (the big numbers are seconds since the epoch):

   recv(1479853346.806, '[37m\x1b[44mStatus: \x1b[7mRun')
   recv(1479853346.826, 'ning\x1b[2;5H\x1b[27mC')
   recv(1479853346.846, 'ommand: \x1b[7m/sbi')
   recv(1479853346.866, 'n/ifconfig -w 15 -W 5\r\n\r')
   recv(1479853346.886, '\n\x1b[27m----------')
   recv(1479853346.906, '------------------------')
   recv(1479853346.926, '----------------')
   recv(1479853346.946, '----------------')
   recv(1479853346.966, '--------------\x1b[4;1H\x1b[m\r')
   recv(1479853346.986, '\n\x1b[m')
   recv(1479853351.815, '\x1b[1;14H\x1b')
   recv(1479853351.828, '[37m\x1b[44')
   recv(1479853351.848, 'm\x1b[7mFinished\x1b[2')

 > Could you wait a few minutes and then look at the ifconfig output again?
 > Guessing the addresses still say TENTATIVE.

 No, they don't.  I changed the test to run ifconfig -a twice, with a
 delay of 120 seconds inbetween.  The first one printed:

 sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
         ec_enabled=0
         address: 00:1b:fc:9e:0f:b4
         media: Ethernet autoselect (10baseT half-duplex)
         status: active
         inet 10.169.0.2/24 broadcast 10.169.0.255 flags 0x1<TENTATIVE>
         inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x2<TENTATIVE> scopeid 0x3

 and the second one printed:

 sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
         ec_enabled=0
         address: 00:1b:fc:9e:0f:b4
         media: Ethernet autoselect (10baseT half-duplex)
         status: active
         inet 10.169.0.2/24 broadcast 10.169.0.255 flags 0x0
         inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x0 scopeid 0x3

 -- 
 Andreas Gustafsson, gson@gson.org

From: Roy Marples <roy@marples.name>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 09:15:09 +0000

 On 23/11/2016 08:33, Andreas Gustafsson wrote:
 > Roy Marples wrote:
 >>>> Do you know if the new ifconfig command did in fact wait for around
 >>>> 15 seconds before running ping?
 > 
 > It waits for about 5 seconds, not 15.  Here's the relevant part
 > of the timestamped log (the big numbers are seconds since the epoch):

 OK.
 That tells me that the address is DETACHED during this time and not
 TENTATIVE.
 In other words, the interface thinks there is no carrier.

 Could you try changing my commit locally from -w 15 -W 5 to -w 15 -W 15
 and see if that helps at all?

 If not, do other numbers help? -w 120 -W 120 is insane, but it's worth
 trying to see if my theory is right.

 If so, we just have to increase the default numbers, and maybe in
 rc.conf as well OR look into why the interface is taking more than 5
 seconds to gain a carrier as sk0 is cabled yes?

 Roy

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 12:07:19 +0200

 Roy Marples wrote:
 > > It waits for about 5 seconds, not 15.  Here's the relevant part
 > > of the timestamped log (the big numbers are seconds since the epoch):
 > 
 > OK.
 > That tells me that the address is DETACHED during this time and not
 > TENTATIVE.

 Shouldn't there be a DETACHED flag in the ifconfig output, then?

 > In other words, the interface thinks there is no carrier.
 > 
 > Could you try changing my commit locally from -w 15 -W 5 to -w 15 -W 15
 > and see if that helps at all?

 Not easily.  The testbed in case is generating public reports that are
 supposed to reflect the true state of -current as a function of the
 CVS source date.  To test something that is not actually -current,
 I'd have to set up a separate instance of the testbed.

 > If not, do other numbers help? -w 120 -W 120 is insane, but it's worth
 > trying to see if my theory is right.
 > 
 > If so, we just have to increase the default numbers, and maybe in
 > rc.conf as well OR look into why the interface is taking more than 5
 > seconds to gain a carrier as sk0 is cabled yes?

 It is cabled to a 10baseT (sic) hub.  I could try a 100baseTX switch
 and see if that makes a difference.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Roy Marples <roy@marples.name>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 10:30:47 +0000

 On 23/11/2016 10:07, Andreas Gustafsson wrote:
 > Roy Marples wrote:
 >>> It waits for about 5 seconds, not 15.  Here's the relevant part
 >>> of the timestamped log (the big numbers are seconds since the epoch):
 >>
 >> OK.
 >> That tells me that the address is DETACHED during this time and not
 >> TENTATIVE.
 > 
 > Shouldn't there be a DETACHED flag in the ifconfig output, then?

 Maybe not.
 Addresses transition from DETACHED -> TENTATIVE when the carrier comes
 up and this triggers DaD.
 If the carrier (or interface) goes down then all addresses are marked
 DETACHED.
 As Joerg indicated earlier this could happen with wireless networking
 very easily.
 Personally I would be inclinded to remove the ping test entirely - the
 fact it passes is not a guarantee it can download sets.

 ifconfig -w 15 -W 5 says wait 5 seconds for DETACHED to clear and 15 for
 TENTATIVE to clear. So if it's waiting for 5 seconds as you say then the
 carrier has been down for at least 5 seconds for some reason.
 rc.d/network was modified a long time ago with this exact same change.

 >> In other words, the interface thinks there is no carrier.
 >>
 >> Could you try changing my commit locally from -w 15 -W 5 to -w 15 -W 15
 >> and see if that helps at all?
 > 
 > Not easily.  The testbed in case is generating public reports that are
 > supposed to reflect the true state of -current as a function of the
 > CVS source date.  To test something that is not actually -current,
 > I'd have to set up a separate instance of the testbed.
 > 
 >> If not, do other numbers help? -w 120 -W 120 is insane, but it's worth
 >> trying to see if my theory is right.
 >>
 >> If so, we just have to increase the default numbers, and maybe in
 >> rc.conf as well OR look into why the interface is taking more than 5
 >> seconds to gain a carrier as sk0 is cabled yes?
 > 
 > It is cabled to a 10baseT (sic) hub.  I could try a 100baseTX switch
 > and see if that makes a difference.

 Please do.

 Roy

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 15:48:16 +0200

 Roy Marples wrote:
 > Personally I would be inclinded to remove the ping test entirely - the
 > fact it passes is not a guarantee it can download sets.

 I agree - not only is the test passing no guarantee that sets can be
 downloaded, but also, the ping test failing is no guarantee that
 downloading the sets will fail.  But I don't think that has anything
 to do with the present issue: the problem is that downloading the sets
 fails, and whether the ping test succeeds, fails, or even exists is
 irrelevant.

 > > It is cabled to a 10baseT (sic) hub.  I could try a 100baseTX switch
 > > and see if that makes a difference.
 > 
 > Please do.

 Done.  It made no difference.  Here's the ifconfig output from immediately
 after the ftp command failed:

 sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
         ec_enabled=0
         address: 00:1b:fc:9e:0f:b4
         media: Ethernet autoselect (100baseTX full-duplex,flowcontrol,rxpause,txpause)
         status: active
         inet 10.169.0.2/24 broadcast 10.169.0.255 flags 0x1<TENTATIVE>
         inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x2<TENTATIVE> scopeid 0x3

 And 120 seconds later:

 sk0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
         ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
         ec_enabled=0
         address: 00:1b:fc:9e:0f:b4
         media: Ethernet autoselect (100baseTX full-duplex,flowcontrol,rxpause,txpause)
         status: active
         inet 10.169.0.2/24 broadcast 10.169.0.255 flags 0x0
         inet6 fe80::21b:fcff:fe9e:fb4%sk0/64 flags 0x0 scopeid 0x3

 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/51531: Recent networking regression affecting installs
Date: Wed, 23 Nov 2016 17:16:06 +0200

 Roy,

 I took packet captures on the server (10.169.0.1) of a successful
 network install (source date 2016.09.16.10.54.45) and an unsuccessful
 one (sorce date 2016.11.22.12.04.35).  Maybe you can glean some
 useful information from them.

 Here are extracts showing the last couple of packets of the TFTP boot
 followed by the DAD, ping test, and first couple of packets of the HTTP
 set download (if present).   The successful one looks like this:

   16:42:58.073675 00:50:fc:fb:10:45 (oui Unknown) > 00:1b:fc:9e:0f:b4 (oui Unknown), ethertype IPv4 (0x0800), length 558: 10.169.0.1.62529 > 10.169.0.2.dict-lookup: UDP, length 516
   16:42:58.087396 00:1b:fc:9e:0f:b4 (oui Unknown) > 00:50:fc:fb:10:45 (oui Unknown), ethertype IPv4 (0x0800), length 60: 10.169.0.2.dict-lookup > 10.169.0.1.62529: UDP, length 5
   16:44:05.360799 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.2 tell 10.169.0.2, length 46
   16:44:13.346801 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.1 tell 10.169.0.2, length 46
   16:44:13.346811 00:50:fc:fb:10:45 (oui Unknown) > 00:1b:fc:9e:0f:b4 (oui Unknown), ethertype ARP (0x0806), length 42: Reply 10.169.0.1 is-at 00:50:fc:fb:10:45 (oui Unknown), length 28
   16:44:13.348603 00:1b:fc:9e:0f:b4 (oui Unknown) > 00:50:fc:fb:10:45 (oui Unknown), ethertype IPv4 (0x0800), length 98: 10.169.0.2 > 10.169.0.1: ICMP echo request, id 19520, seq 0, length 64
   16:44:13.348614 00:50:fc:fb:10:45 (oui Unknown) > 00:1b:fc:9e:0f:b4 (oui Unknown), ethertype IPv4 (0x0800), length 98: 10.169.0.1 > 10.169.0.2: ICMP echo reply, id 19520, seq 0, length 64
   16:44:16.479645 00:1b:fc:9e:0f:b4 (oui Unknown) > 00:50:fc:fb:10:45 (oui Unknown), ethertype IPv4 (0x0800), length 98: 10.169.0.2 > 10.0.1.1: ICMP echo request, id 22346, seq 0, length 64
   16:44:16.480349 00:50:fc:fb:10:45 (oui Unknown) > 00:1b:fc:9e:0f:b4 (oui Unknown), ethertype IPv4 (0x0800), length 98: 10.0.1.1 > 10.169.0.2: ICMP echo reply, id 22346, seq 0, length 64
   16:44:19.104649 00:1b:fc:9e:0f:b4 (oui Unknown) > 00:50:fc:fb:10:45 (oui Unknown), ethertype IPv4 (0x0800), length 78: 10.169.0.2.65535 > 10.169.0.1.http: Flags [S], seq 2279312074, win 32768, options [mss 1460,nop,wscale 3,sackOK,nop,nop,nop,nop,TS val 1 ecr 0], length 0
   16:44:19.104665 00:50:fc:fb:10:45 (oui Unknown) > 00:1b:fc:9e:0f:b4 (oui Unknown), ethertype IPv4 (0x0800), length 78: 10.169.0.1.http > 10.169.0.2.65535: Flags [S.], seq 241726766, ack 2279312075, win 32768, options [mss 1460,nop,wscale 3,nop,nop,TS val 1 ecr 1,sackOK,nop,nop], length 0
   (successful HTTP download follows)

 The unsuccssful one looks like this:

   21:02:02.196177 00:50:fc:fb:10:45 (oui Unknown) > 00:1b:fc:9e:0f:b4 (oui Unknown), ethertype IPv4 (0x0800), length 558: 10.169.0.1.62681 > 10.169.0.2.netadmin: UDP, length 516
   21:02:02.210352 00:1b:fc:9e:0f:b4 (oui Unknown) > 00:50:fc:fb:10:45 (oui Unknown), ethertype IPv4 (0x0800), length 60: 10.169.0.2.netadmin > 10.169.0.1.62681: UDP, length 5
   21:04:44.366636 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.2 tell 0.0.0.0, length 46
   21:04:44.939141 00:1b:fc:9e:0f:b4 (oui Unknown) > 33:33:ff:9e:0f:b4 (oui Unknown), ethertype IPv6 (0x86dd), length 78: :: > ff02::1:ff9e:fb4: ICMP6, neighbor solicitation, who has fe80::21b:fcff:fe9e:fb4, length 24
   21:04:45.772737 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.2 tell 0.0.0.0, length 46
   21:04:47.670958 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.2 tell 0.0.0.0, length 46
   21:04:49.679675 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.2 tell 10.169.0.2, length 46
   21:04:51.688397 00:1b:fc:9e:0f:b4 (oui Unknown) > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.169.0.2 tell 10.169.0.2, length 46
   (no further packets)

 -- 
 Andreas Gustafsson, gson@gson.org

Responsible-Changed-From-To: kern-bug-people->roy
Responsible-Changed-By: gson@NetBSD.org
Responsible-Changed-When: Mon, 12 Dec 2016 16:42:56 +0000
Responsible-Changed-Why:
roy broke it


From: "Roy Marples" <roy@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Tue, 13 Dec 2016 19:03:49 +0000

 Module Name:	src
 Committed By:	roy
 Date:		Tue Dec 13 19:03:49 UTC 2016

 Modified Files:
 	src/usr.sbin/sysinst: net.c

 Log Message:
 ping is not a reliable means of testing if connectivity to download sets
 actually works, so remove it.
 Hopefully fixes PR kern/51531.


 To generate a diff of this commit:
 cvs rdiff -u -r1.22 -r1.23 src/usr.sbin/sysinst/net.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Andreas Gustafsson <gson@gson.org>
To: roy@NetBSD.org
Cc: gnats-bugs@NetBSD.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 10:28:05 +0200

 Roy Marples wrote:
 >  ping is not a reliable means of testing if connectivity to download sets
 >  actually works, so remove it.

 As I said in an earlier comment to this PR, I support this change,
 but I don't think it is related to the present issue.

 >  Hopefully fixes PR kern/51531.

 It did not.  Console log:

   http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.12.13.19.03.49/install.log

 Timestamped log:

   http://www.gson.org/netbsd/bugs/51531/install-2016.12.13.19.03.49-slog.txt

 Some observations based on the timestamped log:

 The "ifconfig -w 15 -W 5" command runs in five seconds.  The ftp
 command then takes some 74 seconds to time out.

 Then, a diagnostic "ifconfig -a" is run, and it shows a TENTATIVE flag
 even though 83 seconds (!) have now elapsed since the "ifconfig
 -w 15 -W 5" started.

 Ten seconds later, a second diagnostic "ifconfig -a" is run, and then,
 the TENTATIVE flag is gone.

 When reading the timestamped log, note that the console output is not
 being read during the 10-second sleep between the two "ifconfig -a"
 commands, so the output from the first "ifconfig -a" only appears in
 the log after the second "ifconfig -a" has been sent, and its timestamp
 is not meaningful.  The timestamps on the "send(..., 'ifconfig -a\n')"
 lines are correct.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: roy@NetBSD.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 10:45:00 +0200

 A short while ago, I wrote:
 > Ten seconds later, a second diagnostic "ifconfig -a" is run, and then,
 > the TENTATIVE flag is gone.

 A correction to the above: the delay between the two diagnostic
 "ifconfig -a" commands is actually 120 seconds, not 10 seconds.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Roy Marples <roy@marples.name>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, gson@gson.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 13:40:50 +0000

 On 2016-12-14 08:30, Andreas Gustafsson wrote:
 > The following reply was made to PR kern/51531; it has been noted by 
 > GNATS.
 > 
 > From: Andreas Gustafsson <gson@gson.org>
 > To: roy@NetBSD.org
 > Cc: gnats-bugs@NetBSD.org
 > Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
 > Date: Wed, 14 Dec 2016 10:28:05 +0200
 > 
 >  Roy Marples wrote:
 >  >  ping is not a reliable means of testing if connectivity to download 
 > sets
 >  >  actually works, so remove it.
 > 
 >  As I said in an earlier comment to this PR, I support this change,
 >  but I don't think it is related to the present issue.
 > 
 >  >  Hopefully fixes PR kern/51531.
 > 
 >  It did not.  Console log:
 > 
 > 
 > http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.12.13.19.03.49/install.log
 > 
 >  Timestamped log:
 > 
 >    
 > http://www.gson.org/netbsd/bugs/51531/install-2016.12.13.19.03.49-slog.txt
 > 
 >  Some observations based on the timestamped log:
 > 
 >  The "ifconfig -w 15 -W 5" command runs in five seconds.  The ftp
 >  command then takes some 74 seconds to time out.
 > 
 >  Then, a diagnostic "ifconfig -a" is run, and it shows a TENTATIVE flag
 >  even though 83 seconds (!) have now elapsed since the "ifconfig
 >  -w 15 -W 5" started.
 > 
 >  Ten seconds later, a second diagnostic "ifconfig -a" is run, and then,
 >  the TENTATIVE flag is gone.
 > 
 >  When reading the timestamped log, note that the console output is not
 >  being read during the 10-second sleep between the two "ifconfig -a"
 >  commands, so the output from the first "ifconfig -a" only appears in
 >  the log after the second "ifconfig -a" has been sent, and its 
 > timestamp
 >  is not meaningful.  The timestamps on the "send(..., 'ifconfig -a\n')"
 >  lines are correct.

 This sounds like one of two things:
 1) there is a timing issue
 2) the link status is flapping

 Because at the point of the ifconfig -w command there are no futher 
 interface changes, nothing should be resetting the PHY at all so one of 
 the above must be true.
 I say this because the maximum length from DETACHED -> TENTATIVE -> 
 ready to go is 10 seconds once the link is active and your logs show 
 it's at TENTATIVE way beyond that which implies link state is flapping 
 or 10 seconds is not 10 seconds on this device.

 Some idea of how to solve this are
 1) Implement RFC4429 and apply it to IPv4 as well
     (needs a toggle like ifconfig bge0 inet 1.2.3.4/24 optimistic)
 2) Add a toggle to sysinst to disable DaD

 I don't suppose you could try swapping the interfacec out with another 
 one?

 Roy

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 18:39:57 +0200

 Roy Marples wrote:
 > This sounds like one of two things:
 > 1) there is a timing issue
 > 2) the link status is flapping
 > 
 > Because at the point of the ifconfig -w command there are no futher 
 > interface changes, nothing should be resetting the PHY at all so one of 
 > the above must be true.
 > I say this because the maximum length from DETACHED -> TENTATIVE -> 
 > ready to go is 10 seconds once the link is active and your logs show 
 > it's at TENTATIVE way beyond that which implies link state is flapping 
 > or 10 seconds is not 10 seconds on this device.

 I just ran another pair of tests.  First, I made the test script run
 "ifconfig -a" in a loop, once per second for 60 seconds, starting two
 seconds after the failed "ftp".  The "ifconfig -a" output had the
 TENTATIVE flag set in the first five runs, and clear in the remaining
 55 runs.

 Then I did a second run, but now with a 30-second delay between the
 failed "ftp" and the first "ifconfig -a" instead of a 2-second delay.
 Now, the output had the TENTATIVE flag set in the first six runs, and
 clear in the remaining 54.

 So, the TENTATIVE flag appears to be set for some 5-6 seconds,
 counting from the time when "ifconfig -a" is first run, *regardless*
 of when that first run takes place.  The only explanation I can think
 of is that it must be the "ifconfig -a" itself that starts the timer,
 despite supposedly being a read-only operation.

 Is it possible that the initial DAD attempt could somehow get stuck
 indefinitely, for example because of the NIC being in an unexpected
 initial state after the netboot, and that the "ifconfig -a" could
 then cause it to come unstuck?

 > Some idea of how to solve this are
 > 1) Implement RFC4429 and apply it to IPv4 as well

 This, too, seems like a worthwhile change for reasons unrelated to the
 present issue, and it might be a viable work-around, but I don't think
 it addresses the root cause.

 >     (needs a toggle like ifconfig bge0 inet 1.2.3.4/24 optimistic)
 > 2) Add a toggle to sysinst to disable DaD

 IMO, it needs to work with the default settings.  Someone trying to
 install NetBSD for the first time and running into this issue is far
 more likely to give up and move on to the next OS candidate than to
 find out what special settings are needed.

 > I don't suppose you could try swapping the interfacec out with another 
 > one?

 I currently have two machines configured for this automated test.
 I already tried the other one, and it behaved the same way.
 Unfortunately, even though they have different motherboards and CPUs,
 they have the same type of on-board Ethernet interface (Marvell Yukon
 Lite) so the possibility remains that this issue could be specific
 to that chip or the sk driver.

 I'll see if I can find a suitable PCI card (this is not as trivial as
 it may sound because it needs to have a working PXE ROM).  Or maybe
 I'll set up a third machine...
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 21:34:08 +0200

 Roy,

 I found an old Intel PCI network card (fxp) with a PXE ROM, and using
 that, the installation succeeded.  So it looks like your DAD changes
 have exposed a problem with the sk driver.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Roy Marples <roy@marples.name>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 20:44:57 +0000

 Hi Andreas

 On 14/12/16 19:34, Andreas Gustafsson wrote:
 > Roy,
 > 
 > I found an old Intel PCI network card (fxp) with a PXE ROM, and using
 > that, the installation succeeded.  So it looks like your DAD changes
 > have exposed a problem with the sk driver.

 Well, that's good and bad news I guess.
 Good news that you can continue testing, bad news that the underlying
 issue isn't resolved.

 So what do we want to do with this bug?
 I think the sysinst changes to date are still valid.

 Close this and make a new ticket?

 Roy

Responsible-Changed-From-To: roy->kern-bug-people
Responsible-Changed-By: gson@NetBSD.org
Responsible-Changed-When: Wed, 14 Dec 2016 21:12:43 +0000
Responsible-Changed-Why:
The root cause of the problem exposed by roy's change appears
to be in sk(4).


From: Andreas Gustafsson <gson@gson.org>
To: Roy Marples <roy@marples.name>
Cc: gnats-bugs@netbsd.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 23:10:16 +0200

 Roy Marples wrote:
 > Well, that's good and bad news I guess.
 > Good news that you can continue testing, bad news that the underlying
 > issue isn't resolved.

 Right.

 > So what do we want to do with this bug?
 > I think the sysinst changes to date are still valid.

 They can stay; removing the ping test is the right thing to do
 regardless, and I suppose the "ifconfig -w 15 -W 5" is correct given
 the current current behavior of ifconfig where it can exit before the
 address being configured is usable.  I do still think that behavior is
 incorrect, but that is a separate issue which should have a separate PR.

 > Close this and make a new ticket?

 I think we should keep this ticket open, consider it a probable sk(4)
 bug, and assign it back to kern-bug-people.
 -- 
 Andreas Gustafsson, gson@gson.org

From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, roy@NetBSD.org, gnats-admin@netbsd.org, 
	netbsd-bugs@netbsd.org, gson@gson.org (Andreas Gustafsson)
Cc: 
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Wed, 14 Dec 2016 17:21:37 -0500

 On Dec 14,  8:50pm, roy@marples.name (Roy Marples) wrote:
 -- Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst

 |  Hi Andreas
 |  
 |  On 14/12/16 19:34, Andreas Gustafsson wrote:
 |  > Roy,
 |  > 
 |  > I found an old Intel PCI network card (fxp) with a PXE ROM, and using
 |  > that, the installation succeeded.  So it looks like your DAD changes
 |  > have exposed a problem with the sk driver.
 |  
 |  Well, that's good and bad news I guess.
 |  Good news that you can continue testing, bad news that the underlying
 |  issue isn't resolved.
 |  
 |  So what do we want to do with this bug?
 |  I think the sysinst changes to date are still valid.
 |  
 |  Close this and make a new ticket?

 Can you try 1.84 of if_sk.c?

 christos

From: Andreas Gustafsson <gson@gson.org>
To: christos@zoulas.com (Christos Zoulas)
Cc: gnats-bugs@NetBSD.org,
    roy@NetBSD.org
Subject: Re: PR/51531 CVS commit: src/usr.sbin/sysinst
Date: Thu, 15 Dec 2016 19:46:53 +0200

 Christos Zoulas wrote:
 > Can you try 1.84 of if_sk.c?

 Done.  I'm still seeing the same strange behavior where running
 ifconfig -a repeatedly shows the TENTATIVE flag set for the first six
 seconds or so and clear thereafter, even though the loop of "ifconfig
 -a" commands is started at a completely arbitrary point in time
 relative to when the interface was first configured or last used.

 Log:

   http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2016/2016.12.15.13.06.08/install.log

 Timestamped log:

   http://www.gson.org/netbsd/bugs/51531/install-2016.12.15.13.06.08.slog.txt

 -- 
 Andreas Gustafsson, gson@gson.org

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.