NetBSD Problem Report #53421
From mark@ecs.vuw.ac.nz Tue Jul 3 11:44:51 2018
Return-Path: <mark@ecs.vuw.ac.nz>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id BEEFD7A1AE
for <gnats-bugs@gnats.NetBSD.org>; Tue, 3 Jul 2018 11:44:51 +0000 (UTC)
Message-Id: <201807031144.w63BilPn011234@downstage.ecs.vuw.ac.nz>
Date: Tue, 3 Jul 2018 23:44:47 +1200 (NZST)
From: mark@ecs.vuw.ac.nz
Reply-To: mark@ecs.vuw.ac.nz
To: gnats-bugs@NetBSD.org
Subject: 8.0_RC2 named has resource issue
X-Send-Pr-Version: 3.95
>Number: 53421
>Category: misc
>Synopsis: 8.0_RC2 named has resource issue
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: misc-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Jul 03 11:45:00 +0000 2018
>Closed-Date: Sun Aug 19 20:58:07 +0000 2018
>Last-Modified: Wed Feb 27 23:55:00 +0000 2019
>Originator: Mark Davies
>Release: NetBSD 8.0_RC2
>Organization:
ECS, Victoria Uni. of Wellington, New Zealand.
>Environment:
System: NetBSD downstage.ecs.vuw.ac.nz 8.0_RC2 NetBSD 8.0_RC2 (GENERIC) #4: Mon Jun 25 12:23:43 NZST 2018 mark@turakirae.ecs.vuw.ac.nz:/local/SAVE/8_64.obj/src/work/8/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
upgrading a reasonably heavily used nameserver machine from 7.1_STABLE to 8.0_RC2
after running for a few hours named will stop answering queries with the following logged
to syslog:
Jul 2 21:48:28 downstage named[22054]: UDP client handler shutting down due to fatal receive error: not enough free resources
restarting named gets it working for another few hours and then repeats.
Don't know which particular "free resources" its complaining about.
Same problem occurs if you use the named binary from 7.1_STABLE on the 8.0_RC2 system
so the issue is presumably in libraries or the kernel.
>How-To-Repeat:
don't know
>Fix:
don't know
>Release-Note:
>Audit-Trail:
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, misc-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: misc/53421: 8.0_RC2 named has resource issue
Date: Tue, 3 Jul 2018 07:57:42 -0400
On Jul 3, 11:45am, mark@ecs.vuw.ac.nz (mark@ecs.vuw.ac.nz) wrote:
-- Subject: misc/53421: 8.0_RC2 named has resource issue
| Jul 2 21:48:28 downstage named[22054]: UDP client handler shutting down due to fatal receive error: not enough free resources
|
| restarting named gets it working for another few hours and then repeats.
| Don't know which particular "free resources" its complaining about.
| Same problem occurs if you use the named binary from 7.1_STABLE on the 8.0_RC2 system
| so the issue is presumably in libraries or the kernel.
Look for ISC_R_NORESOURCES in:
/usr/src/external/bsd/bind/dist/lib/isc/unix/socket.c
and add some syslog messages before the returns. My guess is that it is
ENOBUFS.
christos
From: Takahiro Kambe <taca@back-street.net>
To: christos@zoulas.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: misc/53421: 8.0_RC2 named has resource issue
Date: Wed, 04 Jul 2018 23:22:23 +0900 (JST)
Hi,
In message <20180703115742.E11B017FDA1@rebar.astron.com>
on Tue, 3 Jul 2018 07:57:42 -0400,
christos@zoulas.com (Christos Zoulas) wrote:
> | so the issue is presumably in libraries or the kernel.
>
> Look for ISC_R_NORESOURCES in:
> /usr/src/external/bsd/bind/dist/lib/isc/unix/socket.c
> and add some syslog messages before the returns. My guess is that it is
> ENOBUFS.
I met the same problem from NetBSD 8.0_RC1. And it happens on both
base named and bind-9.10.7 from pkgsrc's named.
AFAIK, no problem with TCP query.
--
Takahiro Kambe <taca@back-street.net>
From: coypu@sdf.org
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: misc/53421
Date: Sat, 18 Aug 2018 13:44:50 +0000
pkgsrc stable:
https://mail-index.netbsd.org/pkgsrc-changes/2018/08/18/msg179718.html
netbsd-8 pullup:
https://releng.netbsd.org/cgi-bin/req-8.cgi?show=972
State-Changed-From-To: open->closed
State-Changed-By: markd@NetBSD.org
State-Changed-When: Sun, 19 Aug 2018 20:58:07 +0000
State-Changed-Why:
Been running for 5 days with 8.0_STABLE including the socket.c patch with no
reoccurance of the problem.
cheers
mark
From: =?UTF-8?B?SmFyb23DrXIgRG9sZcSNZWs=?= <jaromir.dolecek@gmail.com>
To: jmitchel <jmitchel@bigjar.com>
Cc: "gnats-bugs@NetBSD.org" <gnats-bugs@netbsd.org>
Subject: Re: misc/53421: 8.0_RC2 named has resource issue
Date: Sun, 24 Feb 2019 17:15:36 +0100
Yes, it has been fixed on current and fix was pulled up to netbsd-8. I
believe running just newer kernel from 8.0_STABLE should be enough,
the default was changed to not report the error to appliction unless
it specifically toggles an opt.
Jaromir
Le dim. 24 f=C3=A9vr. 2019 =C3=A0 15:43, jmitchel <jmitchel@bigjar.com> a =
=C3=A9crit :
>
> Hello,
>
> I don't know if this has been fixed in netbsd-8, but I found a
> "solution". By compiling the standard bind software with the following
> options, I have kept my name server up since Dec. 4th. Here's the info
> from named -V:
>
> GameofThrones:/var/named# /usr/local/sbin/named -V
> BIND 9.12.3 <id:6c8e92c>
> running on NetBSD i386 8.0 NetBSD 8.0 (GENERIC) #0: Tue Jul 17 14:59:51
> UTC 2018
> mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC
> built by make with '--disable-symtable' '--disable-atomic'
> compiled by GCC 5.5.0
> compiled with OpenSSL version: OpenSSL 1.0.2k 26 Jan 2017
> linked to OpenSSL version: OpenSSL 1.0.2k 26 Jan 2017
> compiled with zlib version: 1.2.10
> linked to zlib version: 1.2.10
> threads support is enabled
>
> GameofThrones:/var/named# uname -a
> NetBSD GameofThrones 8.0 NetBSD 8.0 (GENERIC) #0: Tue Jul 17 14:59:51
> UTC 2018
> mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC i386
>
> And, just in case someone wanted proof that my name server has been up
> this long:
>
> GameofThrones:/var/named# ps awux | grep named
> root 15764 0.0 4.3 62108 44208 ? Ssl 4Dec18 92:48.36
> /usr/local/sbin/named
>
> Now I should note that I'm using forwarders in my named.conf file, but I
> was experiencing the same problem as reported in this bug. Namely named
> would print the following error to the log file and then no longer
> answer UDP requests:
>
> dnslog.txt.72:23-Nov-2018 06:09:23.130 UDP client handler shutting down
> due to fatal receive error: not enough free resources
>
> I hope this helps in some small way.
>
> Thanks,
>
> Jason M.
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: misc/53421: 8.0_RC2 named has resource issue
Date: Sun, 24 Feb 2019 17:25:58 +0100
On Sun, Feb 24, 2019 at 04:20:01PM +0000, Jaromír Dole?ek wrote:
> Yes, it has been fixed on current and fix was pulled up to netbsd-8. I
> believe running just newer kernel from 8.0_STABLE should be enough,
> the default was changed to not report the error to appliction unless
> it specifically toggles an opt.
There also was a specific fix in bind code, which also got pulled up to
netbsd-8 (post 8.0).
Martin
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: misc/53421: 8.0_RC2 named has resource issue
Date: Wed, 27 Feb 2019 23:53:42 +0000
not sent to gnats (PR traffic should be sent to gnats-bugs@)
------
From: jmitchel <jmitchel@bigjar.com>
To: misc-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: misc/53421: 8.0_RC2 named has resource issue
Date: Sun, 24 Feb 2019 08:15:12 -0500
Hello,
I don't know if this has been fixed in netbsd-8, but I found a "solution". By
compiling the standard bind software with the following options, I have kept
my name server up since Dec. 4th. Here's the info from named -V:
GameofThrones:/var/named# /usr/local/sbin/named -V
BIND 9.12.3 <id:6c8e92c>
running on NetBSD i386 8.0 NetBSD 8.0 (GENERIC) #0: Tue Jul 17 14:59:51 UTC
2018 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC
built by make with '--disable-symtable' '--disable-atomic'
compiled by GCC 5.5.0
compiled with OpenSSL version: OpenSSL 1.0.2k 26 Jan 2017
linked to OpenSSL version: OpenSSL 1.0.2k 26 Jan 2017
compiled with zlib version: 1.2.10
linked to zlib version: 1.2.10
threads support is enabled
GameofThrones:/var/named# uname -a
NetBSD GameofThrones 8.0 NetBSD 8.0 (GENERIC) #0: Tue Jul 17 14:59:51 UTC 2018
mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC i386
And, just in case someone wanted proof that my name server has been up this
long:
GameofThrones:/var/named# ps awux | grep named
root 15764 0.0 4.3 62108 44208 ? Ssl 4Dec18 92:48.36
/usr/local/sbin/named
Now I should note that I'm using forwarders in my named.conf file, but I was
experiencing the same problem as reported in this bug. Namely named would
print the following error to the log file and then no longer answer UDP
requests:
dnslog.txt.72:23-Nov-2018 06:09:23.130 UDP client handler shutting down due to
fatal receive error: not enough free resources
I hope this helps in some small way.
Thanks,
Jason M.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.