NetBSD Problem Report #37281

From  Mon Nov  5 03:58:20 2007
Return-Path: <>
Received: from ( [])
	by (Postfix) with ESMTP id AF91063BA4A
	for <>; Mon,  5 Nov 2007 03:58:20 +0000 (UTC)
Message-Id: <>
Date: Sun, 4 Nov 2007 22:58:17 -0500 (EST)
Subject: ip_len byte order in fr_fastroute
X-Send-Pr-Version: 3.95

>Number:         37281
>Category:       kern
>Synopsis:       IPF: ip_len byte order in fr_fastroute
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Nov 05 04:00:01 +0000 2007
>Last-Modified:  Sun Feb 25 18:24:24 +0000 2018
>Originator:     Ken Raeburn
>Release:        NetBSD 4.0_RC2
System: NetBSD 4.0_RC2 NetBSD 4.0_RC2 (GENERIC) #0: Mon Oct 1 20:52:07 PDT 2007 builds@wb34:/home/builds/ab/netbsd-4-0-RC2/i386/200710012132Z-obj/home/builds/ab/netbsd-4-0-RC2/src/sys/arch/i386/compile/GENERIC i386
Architecture: i386
Machine: i386

My gateway machine at home has two ethernet links active, one to my
local network and one to my ISP.  There's also a tunnel to a server
that routes traffic for several globally-reachable addresses to me.

I'm concerned about the ISP blocking outgoing packets with source
addresses it hasn't assigned.  Using ipfilter rules like "pass out
quick on tlp2 to gif4 from <subnet> to any" to get around ISP
filtering helps with that, or did up through NetBSD 2.0.  When I
upgraded to 3.1 (and shortly thereafter to 4.0RC2), the tunnel stopped
working; much of the outgoing traffic was corrupted.  It turns out
that the corruption seems to be very specific: The length field in the
encapsulated packet is in host byte order rather than network byte

Some hardware watchpoint hacks show that fr_fastroute is restoring the
host byte ordering that was in use when it was called, around line
1252 in 4.0RC2:

     	if (i) {
/* --> */    ip->ip_len = ntohs(ip->ip_len);
       	     ip->ip_off = ntohs(ip->ip_off);

The call stack is:


In fr_check_wrapper, the length and offset fields are switched to host
order, and switched back after fr_check returns, if it doesn't return
an error or zero out the mbuf pointer.

In fr_fastroute, we switch back to network order, call the output
routine (gif_output enqueues a packet which includes a reference to
the original packet data), and then restore the data to host byte
order (before the gif soft interrupt service routine has sent the
encapsulated packet).

(See also the "ip_len byte order in fr_fastroute" thread started
November 4 in .)


Set up a gif tunnel, use ipfilter force some traffic to use the
tunnel.  It may need to be in a machine that is already acting as a
router between multiple ethernets, and the traffic to be corrupted
might need to originate on another local machine.

Look at tcpdump on the interface where outgoing tunnel traffic would
be seen.  I saw lots of "#### bytes missing" reports, where the
difference is the difference between the host-order and net-order
representations of the length.

Also, Hans Rosenfeld says he's seeing similar problems, and referred
to an earlier report (on current-users in early 2005) of what sounds
like the same or a similar problem, in connection with an ipfilter
update and use of NAT.  (I have NAT configured as well, but use a
tunnel instead for the particular traffic where I've been noticing the

Meanwhile, Tracy says (in a conversation over Zephyr several days ago)
that an acquaintance of hers is using gif tunnels but normal routing
tables to get traffic into them, and isn't seeing this problem.



Possible workarounds:
 - hack gif output routines to force correct byte order
 - copy packet in gif_output (might degrade performance)
 - check and see if the ISP isn't actually blocking such traffic these days :-)
   and stop sending through the tunnel if that's the case
 - use NAT for more things (no good for servers, or protocols like Zephyr that
   set up listening UDP sockets locally to receive from multiple servers)




NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD:,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.