NetBSD Problem Report #38773

From martin@duskware.de  Wed May 28 00:14:55 2008
Return-Path: <martin@duskware.de>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by narn.NetBSD.org (Postfix) with ESMTP id 0A7E463B8BC
	for <gnats-bugs@gnats.netbsd.org>; Wed, 28 May 2008 00:14:55 +0000 (UTC)
Message-Id: <20080527233449.834C163B8BC@narn.NetBSD.org>
Date: Tue, 27 May 2008 23:34:49 +0000 (UTC)
From: paul@whooppee.com
Reply-To: paul@whooppee.com
To: netbsd-bugs-owner@NetBSD.org
Subject: ipf/ipnat broken in 4.99.63
X-Send-Pr-Version: www-1.0

>Number:         38773
>Category:       kern
>Synopsis:       ipf/ipnat broken in 4.99.63
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    tsutsui
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed May 28 00:15:00 +0000 2008
>Closed-Date:    Sun Apr 19 16:36:25 +0000 2009
>Last-Modified:  Sun Apr 19 16:36:25 +0000 2009
>Originator:     Paul Goyette
>Release:        4.99.63
>Organization:
>Environment:
NetBSD quicky.whooppee.com 4.99.63 NetBSD 4.99.63 (QUICKY (ASUS M2N32 WS) 2008-05-23 19:09:17) #4: Sat May 24 05:04:53 PDT 2008  paul@quicky.whooppee.com:/build/obj/amd64/sys/arch/amd64/compile/QUICKY amd64

>Description:
The recent import of ipf appears broken.

With no ipf rules defined, but with ipnat enabled, I am unable to ssh from another machine into the 4.99.63 box.  SSH -d shows that the TCP session is established, but fails during the key exchange.  I can open an ftp session but unable to perform an ls.  ICMP pings are properly responded to.  NFS mounts of remote file systems also work, but any attempt to to a 'df' or to otherwise access the mounted directory fails.

I'm suspecting a problem with packet size, but am unable to confirm.  I have had to revert this machine back to 4.99.62 (from May 14th) since I cannot disable ipnat (my VoIP gateway needs an IP address).
>How-To-Repeat:
See description.
>Fix:

>Release-Note:

>Audit-Trail:
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: kern-bug-people@NetBSD.org, gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: kern/38773: ipf/ipnat broken in 4.99.63
Date: Wed, 28 May 2008 21:18:33 +0200

 On Wed, May 28, 2008 at 12:15:00AM +0000, paul@whooppee.com wrote:
 > >Description:
 > The recent import of ipf appears broken.
 > 
 > With no ipf rules defined, but with ipnat enabled, I am unable to ssh from another machine into the 4.99.63 box.  SSH -d shows that the TCP session is established, but fails during the key exchange.  I can open an ftp session but unable to perform an ls.  ICMP pings are properly responded to.  NFS mounts of remote file systems also work, but any attempt to to a 'df' or to otherwise access the mounted directory fails.
 > 
 > I'm suspecting a problem with packet size, but am unable to confirm.  I have had to revert this machine back to 4.99.62 (from May 14th) since I cannot disable ipnat (my VoIP gateway needs an IP address).

 FWIW, my home gateway is running 4.99.63 with the new ipfilter and it's
 no worse than the ipfilter that was in 4.99.62 (it's even sightly better).
 I can reach it via ssh, ftp or http without troubles.
 I'm also using ipf+ipnat.

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/38773: ipf/ipnat broken in 4.99.63
Date: Thu, 29 May 2008 20:06:32 -0700 (PDT)

 As an additional data point, I have verified that the problems noted in 
 this PR are due to the recent update to ipf.

 I reverted src/sys/dist/ipf/ to the state as of 2008-05-19 07:00 UTC 
 while leaving the rest of src/sys/ is up to date as of  2008-05-28 
 17:35:53 UTC.  A kernel built with the exact same config file works 
 perfectly.


 ----------------------------------------------------------------------
 |   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:   |
 | Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul@whooppee.com   |
 | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette@juniper.net |
 ----------------------------------------------------------------------

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/38773: ipf/ipnat broken in 4.99.63
Date: Fri, 30 May 2008 05:42:01 -0700 (PDT)

 More info in an attempt to narrow things down...

 1. When the problem is occurring, a tcpdump shows no traffic in or
     out of the box.

 2. This is specific to ipnat.  Stopping ipnat via `/etc/rc.d/ipnat
     stop' lets things move, and previously-stalled network activity
     resumes.  Restarting ipnat brings the problem back immediately.

 3. The simplest way for me to cause the "hang" is to log in on a
     wsconsole and use `ls /home/paul' (where /home is NFS-mounted
     from a remote system on the "public network" - see diagram below).

 4. When the ls command hangs, running 'ps -owchan' from another
     session shows that ls is waiting for netio.

 5. The problem appears to affect only local traffic (ie, to or from
     the nat host itself).  Nat'd traffic works just fine.

 6. I'm still quite confused over how I can successfully nfs-mount
     the remote file system, yet once it is mounted I cannot do the
     'ls' command!  And some other stuff still works:
  	* I can establish an ftp connection from the public network,
  	  and even transfer some data.
  	* ICMP pings still work, even with a large packet size.
  	* nntp traffic seems unaffected to multiple sources

 Since this seems to almost certainly be a problem with ipnat, here
 is my /etc/ipnat.conf file:

  	map re0 192.168.2.0/25 -> 0/32 proxy port ftp ftp/tcp
  	map re0 192.168.2.0/25 -> 0/32 portmap tcp/udp 40000:60000
  	map re0 192.168.2.0/25 -> 0/32

  	                       +--------+
  	Public Net        re0  |        | nfe0      Private Net
  	<----------------------| SPEEDY |--------------------->
  	     66.92.186.133/29  |        | 192.168.2.250/24
  	                       +--------+

 The "private net" is divided into two halves, with the lower half
 (.0 through .127 - my VoIP gateway and several WiFi stations) being
 allowed to access the internet via nat and the upper half (.128
 through .255) being totally private (that's where my printer lives).

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/38773: ipf/ipnat broken in 4.99.63
Date: Fri, 30 May 2008 07:38:21 -0700 (PDT)

 On Fri, 30 May 2008, Paul Goyette wrote:

 > 6. I'm still quite confused over how I can successfully nfs-mount
 >   the remote file system, yet once it is mounted I cannot do the
 >   'ls' command!  And some other stuff still works:
 > 	* I can establish an ftp connection from the public network,
 > 	  and even transfer some data.
 > 	* ICMP pings still work, even with a large packet size.
 > 	* nntp traffic seems unaffected to multiple sources

 One more interesting data point!  This seems to be specific to one 
 particular NFS remote file system.

 My NetBSD build environment is mounted from host X, and I can access
 it without any problems.  I can 'ls /build' or 'ls /build/src' and
 everything is fine.

 However, my home directories are mounted from host Y.  I can 'ls
 /home' and successfully get a list of the five user directories.
 But if I 'ls /home/paul' (which contains some 60+ entries) it just
 hangs.  And once it hangs, it is hung solid and I can no longer
 'ls /home' either.  Further, I cannot 'umount /home' - not even
 with -f.

Responsible-Changed-From-To: kern-bug-people->ipf-bug-people
Responsible-Changed-By: darrenr@NetBSD.org
Responsible-Changed-When: Thu, 01 Jan 2009 04:14:40 +0000
Responsible-Changed-Why:


State-Changed-From-To: open->feedback
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Fri, 17 Apr 2009 12:42:45 +0900
State-Changed-Why:
Could you please try the following change as mentioned PR kern/41074?
http://ipfilter.cvs.sourceforge.net/viewvc/ipfilter/ipfilter/ip_nat.c?r1=1.2.2.47&r2=1.2.2.48&pathrev=v4-1-RELEASE
It fixes changes applied in a window you mentioned in this PR.


From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@NetBSD.org
Cc: ipf-bug-people@netbsd.org, netbsd-bugs@netbsd.org, gnats-admin@netbsd.org, 
    tsutsui@NetBSD.org
Subject: Re: kern/38773 (ipf/ipnat broken in 4.99.63)
Date: Fri, 17 Apr 2009 12:07:41 -0700 (PDT)

 On Fri, 17 Apr 2009, tsutsui@NetBSD.org wrote:

 > Could you please try the following change as mentioned PR kern/41074?
 > http://ipfilter.cvs.sourceforge.net/viewvc/ipfilter/ipfilter/ip_nat.c?r1=1.2.2.47&r2=1.2.2.48&pathrev=v4-1-RELEASE
 > It fixes changes applied in a window you mentioned in this PR.

 I've attempted to install that patch, but my machine crashes immediately 
 during startup.  I'm trying to determine if it is patch-related or if it 
 is something else.


 -------------------------------------------------------------------------
 |   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:      |
 | Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul at whooppee.com   |
 | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
 | Kernel Developer |                          | pgoyette at netbsd.org  |
 -------------------------------------------------------------------------

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@NetBSD.org
Cc: ipf-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/38773 (ipf/ipnat broken in 4.99.63)
Date: Fri, 17 Apr 2009 12:34:50 -0700 (PDT)

 On Fri, 17 Apr 2009, Paul Goyette wrote:

 > On Fri, 17 Apr 2009, tsutsui@NetBSD.org wrote:
 >
 > > Could you please try the following change as mentioned PR kern/41074?
 > > http://ipfilter.cvs.sourceforge.net/viewvc/ipfilter/ipfilter/ip_nat.c?r1=1.2.2.47&r2=1.2.2.48&pathrev=v4-1-RELEASE
 > > It fixes changes applied in a window you mentioned in this PR.
 >
 > I've attempted to install that patch, but my machine crashes immediately
 > during startup.  I'm trying to determine if it is patch-related or if it
 > is something else.

 Good news - a clean install of a clean kernel with a clean patch no 
 longer crashes!

 More good news - I can turn on ipnat and still have my UDP-mounted NFS 
 file systems working.

 Looks like "Problem solved"


 -------------------------------------------------------------------------
 |   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:      |
 | Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul at whooppee.com   |
 | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
 | Kernel Developer |                          | pgoyette at netbsd.org  |
 -------------------------------------------------------------------------

From: Izumi Tsutsui <tsutsui@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/38773 CVS commit: src/sys/dist/ipf/netinet
Date: Sat, 18 Apr 2009 11:19:10 +0000

 Module Name:	src
 Committed By:	tsutsui
 Date:		Sat Apr 18 11:19:09 UTC 2009

 Modified Files:
 	src/sys/dist/ipf/netinet: ip_nat.c

 Log Message:
 Pull a fix for ipnat from upstream as per info from darrenr@:
  2031730 4.1.31 Nat drops fragmented packets after the first
  http://ipfilter.cvs.sourceforge.net/viewvc/ipfilter/ipfilter/ip_nat.c#rev1.2.2.48

 Fixes problems on UDP NFS with ipnat as mentioned in PR kern/38773 and
 PR kern/41074.  Tested on several slow NFS clients and an i386 server
 running ipnat.

 Should be pulled up to 5.0.


 To generate a diff of this commit:
 cvs rdiff -u -r1.38 -r1.39 src/sys/dist/ipf/netinet/ip_nat.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: ipf-bug-people->tsutsui
Responsible-Changed-By: tsutsui@NetBSD.org
Responsible-Changed-When: Sun, 19 Apr 2009 23:17:10 +0900
Responsible-Changed-Why:
I've committed the fix.


State-Changed-From-To: feedback->pending-pullups
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Sun, 19 Apr 2009 23:17:10 +0900
State-Changed-Why:
netbsd-5 ticket #710


From: Soren Jacobsen <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/38773 CVS commit: [netbsd-5] src/sys/dist/ipf/netinet
Date: Sun, 19 Apr 2009 15:50:50 +0000

 Module Name:	src
 Committed By:	snj
 Date:		Sun Apr 19 15:50:50 UTC 2009

 Modified Files:
 	src/sys/dist/ipf/netinet [netbsd-5]: ip_nat.c

 Log Message:
 Pull up following revision(s) (requested by tsutsui in ticket #710):
 	sys/dist/ipf/netinet/ip_nat.c: revision 1.39
 Pull a fix for ipnat from upstream as per info from darrenr@:
  2031730 4.1.31 Nat drops fragmented packets after the first
  http://ipfilter.cvs.sourceforge.net/viewvc/ipfilter/ipfilter/ip_nat.c#rev1.2.2.48
 Fixes problems on UDP NFS with ipnat as mentioned in PR kern/38773 and
 PR kern/41074.  Tested on several slow NFS clients and an i386 server
 running ipnat.


 To generate a diff of this commit:
 cvs rdiff -u -r1.38 -r1.38.4.1 src/sys/dist/ipf/netinet/ip_nat.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Mon, 20 Apr 2009 01:36:25 +0900
State-Changed-Why:
pullup done.


>Unformatted:

Home	PR Database Search