NetBSD Problem Report #50511
From cfuhrman@fuhrwerks.com Thu Dec 10 19:44:05 2015
Return-Path: <cfuhrman@fuhrwerks.com>
Received: from mail.netbsd.org (mail.NetBSD.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 407F7A6662
for <gnats-bugs@gnats.NetBSD.org>; Thu, 10 Dec 2015 19:44:05 +0000 (UTC)
Message-Id: <20151210182900.A203D6C245@mail.fuhrwerks.com>
Date: Thu, 10 Dec 2015 13:29:00 -0500 (EST)
From: cfuhrman@pobox.com
Reply-To: cfuhrman@pobox.com
To: gnats-bugs@NetBSD.org
Subject: npf fails to load tree file above certain size
X-Send-Pr-Version: 3.95
>Number: 50511
>Category: kern
>Synopsis: npf fails to load tree file above ~473 entries
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Thu Dec 10 19:45:00 +0000 2015
>Closed-Date: Fri Dec 07 15:05:54 +0000 2018
>Last-Modified: Fri Dec 07 15:05:54 +0000 2018
>Originator: Christopher M. Fuhrman
>Release: NetBSD 7.0
>Organization:
>Environment:
System: NetBSD vc75.vc.panix.com 7.0 NetBSD 7.0 (PANIX-VC) #1: Tue Nov 10 17:40:17 EST 2015 root@juggler.panix.com:/misc/obj64/misc/devel/netbsd/7.0/src/sys/arch/amd64/compile/PANIX-VC amd64
Architecture: x86_64
Machine: amd64
>Description:
Recently, I switched from using OpenBSD pf to NetBSD's spiffy new npf
packet filter on my NetBSD vHost. As part of my configuration, I am
loading a file containing IPv4 address ranges as follows:
table <countries> type tree file "/var/db/npf_tables/countries.txt"
What I've determined is that if the file is above a certain length
(around 473 entries), then npf will fail with the following error:
# npfctl reload
npfctl: npfctl_config_send: Invalid argument
Smaller files load okay.
This behavior has been confirmed with both a Xen-based NetBSD domU and
a VMware Fusion instance running on my Mac (running GENERIC).
**IMPORTANT**
This bug is applicable to the /size of the file/ getting loaded by npf
*not* the size of the table itself. In other words, if I did a
for-loop and loaded each entry via `npfctl table add ...`, then things
work as expected.
>How-To-Repeat:
Create the following:
1. A tree-hash table file containing over 475 entries (give-or-take)
2. An npf.conf(5) file that loads the above file.
3. Load the file via npfctl(8)
If you need a copy of my npf.conf file, please let me know and I can
send it via email in private.
>Fix:
A temporary workaround is to load each entry in a for-loop although
this is not ideal.
>Release-Note:
>Audit-Trail:
From: "David H. Gutteridge" <dhgutteridge@sympatico.ca>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Tue, 16 Aug 2016 20:01:23 -0400
It's possible the following post is relevant to this problem:
http://mail-index.netbsd.org/netbsd-users/2016/04/20/msg018363.html
Have you tried with a table type other than "tree"?
Dave
From: Geoff Wing <gcw@pobox.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Wed, 17 Aug 2016 18:04:11 +1000
This is hitting an arbitrary limit in proplib (65536 byte limit on an
ioctl) in _prop_object_copyin()
from common/lib/libprop/prop_kern.c:
/* Arbitrary limit ioctl input to 64KB */
unsigned int prop_object_copyin_limit = 65536;
On my amd64 system each table line is taking around 121 bytes. Tables
with, say, 4400 lines want over 1/2 MB.
Patch below shows the total size it tries to copy.
Index: common/lib/libprop/prop_kern.c
===================================================================
RCS file: /cvsroot/src/common/lib/libprop/prop_kern.c,v
retrieving revision 1.19
diff -u -r1.19 prop_kern.c
--- common/lib/libprop/prop_kern.c 11 May 2015 16:48:34 -0000 1.19
+++ common/lib/libprop/prop_kern.c 17 Aug 2016 07:55:06 -0000
@@ -407,8 +407,10 @@
char *buf;
int error;
- if (pref->pref_len >= prop_object_copyin_limit)
+ if (pref->pref_len >= prop_object_copyin_limit) {
+ printf("_prop_object_copyin: requested object size (%u) above limit\n", (unsigned int) pref->pref_len);
return EINVAL;
+ }
/*
* Allocate an extra byte so we can guarantee NUL-termination.
From: Christopher Fuhrman <cfuhrman@pobox.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Wed, 17 Aug 2016 14:05:09 -0700
Unfortunately, the table I'm trying to load includes bitmasks which only
work with tree-type tables per npf.conf(5).
--
Christopher M. Fuhrman
cfuhrman@pobox.com
On Wed, Aug 17, 2016, at 01:05 AM, Geoff Wing wrote:
> The following reply was made to PR kern/50511; it has been noted by
> GNATS.
>
> From: Geoff Wing <gcw@pobox.com>
> To: gnats-bugs@netbsd.org
> Cc:
> Subject: Re: kern/50511 (npf fails to load tree file above certain size)
> Date: Wed, 17 Aug 2016 18:04:11 +1000
>
> This is hitting an arbitrary limit in proplib (65536 byte limit on an
> ioctl) in _prop_object_copyin()
>
> from common/lib/libprop/prop_kern.c:
> /* Arbitrary limit ioctl input to 64KB */
> unsigned int prop_object_copyin_limit = 65536;
>
> On my amd64 system each table line is taking around 121 bytes. Tables
> with, say, 4400 lines want over 1/2 MB.
>
> Patch below shows the total size it tries to copy.
>
> Index: common/lib/libprop/prop_kern.c
> ===================================================================
> RCS file: /cvsroot/src/common/lib/libprop/prop_kern.c,v
> retrieving revision 1.19
> diff -u -r1.19 prop_kern.c
> --- common/lib/libprop/prop_kern.c 11 May 2015 16:48:34 -0000
> 1.19
> +++ common/lib/libprop/prop_kern.c 17 Aug 2016 07:55:06 -0000
> @@ -407,8 +407,10 @@
> char *buf;
> int error;
>
> - if (pref->pref_len >= prop_object_copyin_limit)
> + if (pref->pref_len >= prop_object_copyin_limit) {
> + printf("_prop_object_copyin: requested object size (%u)
> above limit\n", (unsigned int) pref->pref_len);
> return EINVAL;
> + }
>
> /*
> * Allocate an extra byte so we can guarantee NUL-termination.
>
From: Geoff Wing <gcw@pobox.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/50511 (npf fails to load tree file above certain size)
Date: Thu, 18 Aug 2016 16:35:58 +1000
You can change it to allow large allocations but I presume there is
potential for kernel memory exhaustion by user programs.
Index: common/lib/libprop/prop_kern.c
===================================================================
RCS file: /cvsroot/src/common/lib/libprop/prop_kern.c,v
retrieving revision 1.19
diff -u -r1.19 prop_kern.c
--- common/lib/libprop/prop_kern.c 11 May 2015 16:48:34 -0000 1.19
+++ common/lib/libprop/prop_kern.c 18 Aug 2016 06:31:56 -0000
@@ -408,7 +408,7 @@
int error;
if (pref->pref_len >= prop_object_copyin_limit)
- return EINVAL;
+ printf("_prop_object_copyin: large requested object size (%u)\n", (unsigned int) pref->pref_len);
/*
* Allocate an extra byte so we can guarantee NUL-termination.
State-Changed-From-To: open->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sun, 10 Dec 2017 00:15:46 +0000
State-Changed-Why:
The limitation in proplib has been fixed in netbsd-8.
State-Changed-From-To: feedback->closed
State-Changed-By: maxv@NetBSD.org
State-Changed-When: Fri, 07 Dec 2018 15:05:54 +0000
State-Changed-Why:
Also pulled up to NetBSD-7 last month. So this PR can be closed.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.