NetBSD Problem Report #50511

From cfuhrman@fuhrwerks.com  Thu Dec 10 19:44:05 2015
Return-Path: <cfuhrman@fuhrwerks.com>
Received: from mail.netbsd.org (mail.NetBSD.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 407F7A6662
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 10 Dec 2015 19:44:05 +0000 (UTC)
Message-Id: <20151210182900.A203D6C245@mail.fuhrwerks.com>
Date: Thu, 10 Dec 2015 13:29:00 -0500 (EST)
From: cfuhrman@pobox.com
Reply-To: cfuhrman@pobox.com
To: gnats-bugs@NetBSD.org
Subject: npf fails to load tree file above certain size
X-Send-Pr-Version: 3.95

>Number:         50511
>Category:       kern
>Synopsis:       npf fails to load tree file above ~473 entries
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Dec 10 19:45:00 +0000 2015
>Closed-Date:    Fri Dec 07 15:05:54 +0000 2018
>Last-Modified:  Fri Dec 07 15:05:54 +0000 2018
>Originator:     Christopher M. Fuhrman
>Release:        NetBSD 7.0
>Organization:

>Environment:
System: NetBSD vc75.vc.panix.com 7.0 NetBSD 7.0 (PANIX-VC) #1: Tue Nov 10 17:40:17 EST 2015 root@juggler.panix.com:/misc/obj64/misc/devel/netbsd/7.0/src/sys/arch/amd64/compile/PANIX-VC amd64
Architecture: x86_64
Machine: amd64
>Description:

Recently, I switched from using OpenBSD pf to NetBSD's spiffy new npf
packet filter on my NetBSD vHost.  As part of my configuration, I am
loading a file containing IPv4 address ranges as follows:

  table <countries> type tree file "/var/db/npf_tables/countries.txt"

What I've determined is that if the file is above a certain length
(around 473 entries), then npf will fail with the following error:

  # npfctl reload
  npfctl: npfctl_config_send: Invalid argument

Smaller files load okay.

This behavior has been confirmed with both a Xen-based NetBSD domU and
a VMware Fusion instance running on my Mac (running GENERIC).

**IMPORTANT**

This bug is applicable to the /size of the file/ getting loaded by npf
*not* the size of the table itself.  In other words, if I did a
for-loop and loaded each entry via `npfctl table add ...`, then things
work as expected.

>How-To-Repeat:

Create the following:

 1. A tree-hash table file containing over 475 entries (give-or-take)
 2. An npf.conf(5) file that loads the above file.
 3. Load the file via npfctl(8)

If you need a copy of my npf.conf file, please let me know and I can
send it via email in private.

>Fix:

A temporary workaround is to load each entry in a for-loop although
this is not ideal.

>Release-Note:

>Audit-Trail:
From: "David H. Gutteridge" <dhgutteridge@sympatico.ca>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Tue, 16 Aug 2016 20:01:23 -0400

 It's possible the following post is relevant to this problem:
 http://mail-index.netbsd.org/netbsd-users/2016/04/20/msg018363.html

 Have you tried with a table type other than "tree"?

 Dave

From: Geoff Wing <gcw@pobox.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Wed, 17 Aug 2016 18:04:11 +1000

 This is hitting an arbitrary limit in proplib (65536 byte limit on an
 ioctl) in _prop_object_copyin()

 from common/lib/libprop/prop_kern.c:
 	/* Arbitrary limit ioctl input to 64KB */
 	unsigned int prop_object_copyin_limit = 65536;

 On my amd64 system each table line is taking around 121 bytes.  Tables
 with, say, 4400 lines want over 1/2 MB.

 Patch below shows the total size it tries to copy.

 Index: common/lib/libprop/prop_kern.c
 ===================================================================
 RCS file: /cvsroot/src/common/lib/libprop/prop_kern.c,v
 retrieving revision 1.19
 diff -u -r1.19 prop_kern.c
 --- common/lib/libprop/prop_kern.c	11 May 2015 16:48:34 -0000	1.19
 +++ common/lib/libprop/prop_kern.c	17 Aug 2016 07:55:06 -0000
 @@ -407,8 +407,10 @@
  	char *buf;
  	int error;

 -	if (pref->pref_len >= prop_object_copyin_limit)
 +	if (pref->pref_len >= prop_object_copyin_limit) {
 +		printf("_prop_object_copyin: requested object size (%u) above limit\n", (unsigned int) pref->pref_len);
  		return EINVAL;
 +	}

  	/*
  	 * Allocate an extra byte so we can guarantee NUL-termination.

From: Christopher Fuhrman <cfuhrman@pobox.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Wed, 17 Aug 2016 14:05:09 -0700

 Unfortunately, the table I'm trying to load includes bitmasks which only
 work with tree-type tables per npf.conf(5).

 --
 Christopher M. Fuhrman
 cfuhrman@pobox.com

 On Wed, Aug 17, 2016, at 01:05 AM, Geoff Wing wrote:
 > The following reply was made to PR kern/50511; it has been noted by
 > GNATS.
 > 
 > From: Geoff Wing <gcw@pobox.com>
 > To: gnats-bugs@netbsd.org
 > Cc: 
 > Subject: Re: kern/50511 (npf fails to load tree file above certain size)
 > Date: Wed, 17 Aug 2016 18:04:11 +1000
 > 
 >  This is hitting an arbitrary limit in proplib (65536 byte limit on an
 >  ioctl) in _prop_object_copyin()
 >  
 >  from common/lib/libprop/prop_kern.c:
 >  	/* Arbitrary limit ioctl input to 64KB */
 >  	unsigned int prop_object_copyin_limit = 65536;
 >  
 >  On my amd64 system each table line is taking around 121 bytes.  Tables
 >  with, say, 4400 lines want over 1/2 MB.
 >  
 >  Patch below shows the total size it tries to copy.
 >  
 >  Index: common/lib/libprop/prop_kern.c
 >  ===================================================================
 >  RCS file: /cvsroot/src/common/lib/libprop/prop_kern.c,v
 >  retrieving revision 1.19
 >  diff -u -r1.19 prop_kern.c
 >  --- common/lib/libprop/prop_kern.c      11 May 2015 16:48:34 -0000     
 >  1.19
 >  +++ common/lib/libprop/prop_kern.c      17 Aug 2016 07:55:06 -0000
 >  @@ -407,8 +407,10 @@
 >   	char *buf;
 >   	int error;
 >   
 >  -       if (pref->pref_len >= prop_object_copyin_limit)
 >  +       if (pref->pref_len >= prop_object_copyin_limit) {
 >  +               printf("_prop_object_copyin: requested object size (%u)
 >  above limit\n", (unsigned int) pref->pref_len);
 >   		return EINVAL;
 >  +       }
 >   
 >   	/*
 >   	 * Allocate an extra byte so we can guarantee NUL-termination.
 >  

From: Geoff Wing <gcw@pobox.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Thu, 18 Aug 2016 16:35:58 +1000

 You can change it to allow large allocations but I presume there is
 potential for kernel memory exhaustion by user programs.  

 Index: common/lib/libprop/prop_kern.c
 ===================================================================
 RCS file: /cvsroot/src/common/lib/libprop/prop_kern.c,v
 retrieving revision 1.19
 diff -u -r1.19 prop_kern.c
 --- common/lib/libprop/prop_kern.c	11 May 2015 16:48:34 -0000	1.19
 +++ common/lib/libprop/prop_kern.c	18 Aug 2016 06:31:56 -0000
 @@ -408,7 +408,7 @@
  	int error;

  	if (pref->pref_len >= prop_object_copyin_limit)
 -		return EINVAL;
 +		printf("_prop_object_copyin: large requested object size (%u)\n", (unsigned int) pref->pref_len);

  	/*
  	 * Allocate an extra byte so we can guarantee NUL-termination.

State-Changed-From-To: open->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sun, 10 Dec 2017 00:15:46 +0000
State-Changed-Why:
The limitation in proplib has been fixed in netbsd-8.


State-Changed-From-To: feedback->closed
State-Changed-By: maxv@NetBSD.org
State-Changed-When: Fri, 07 Dec 2018 15:05:54 +0000
State-Changed-Why:
Also pulled up to NetBSD-7 last month. So this PR can be closed.


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.