NetBSD Problem Report #48098
From www@NetBSD.org Tue Jul 30 07:47:24 2013
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 7200170FB2
for <gnats-bugs@gnats.NetBSD.org>; Tue, 30 Jul 2013 07:47:24 +0000 (UTC)
Message-Id: <20130730074723.075E2710AC@mollari.NetBSD.org>
Date: Tue, 30 Jul 2013 07:47:23 +0000 (UTC)
From: marcotte@panix.com
Reply-To: marcotte@panix.com
To: gnats-bugs@NetBSD.org
Subject: panic: kernel diagnostic assertion "cred != NULL": sys/kern/kern_auth.c
X-Send-Pr-Version: www-1.0
>Number: 48098
>Category: kern
>Synopsis: panic: kernel diagnostic assertion "cred != NULL": sys/kern/kern_auth.c
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Jul 30 07:50:00 +0000 2013
>Closed-Date: Sun May 01 05:18:07 +0000 2016
>Last-Modified: Sun May 01 05:18:07 +0000 2016
>Originator: Brian Marcotte
>Release: 6.1
>Organization:
Public Access Networks, Corp.
>Environment:
NetBSD dw6.panix.com 6.1 NetBSD 6.1 (PANIX-XEN-WEB-DEBUG) #1: Tue Jul 23 23:10:42 EDT 2013 root@juggler.panix.com:/misc/obj/misc/devel/netbsd/6.1/src/sys/arch/i386/compile/PANIX-XEN-WEB-DEBUG i386
>Description:
We're occasionally seeing these panics:
panic: kernel diagnostic assertion "cred != NULL" failed: file "/misc/devel/netbsd/6.1/src/sys/kern/kern_auth.c", line 300
cpu3: Begin traceback...
kern_assert(c03e74d4,c03e7515,c03f5ff1,c03f53f0,12c,3,e2433c14,c030f30d,0,12355bc6) at netbsd:kern_assert+0x23
kauth_cred_geteuid(0,12355bc6,25ef,e50854a6,2500,0,0,e2433b2c,c02b017e,2) at netbsd:kauth_cred_geteuid+0x43
sysctl_net_inet_tcp_ident(e2433c94,0,bf7fdde4,e2433cb4,bf7fdf4c,100,e2433c84,ca35cd40,c86f58a0,4) at netbsd:sysctl_net_inet_tcp_ident+0x2fd
sysctl_dispatch(e2433c84,4,bf7fdde4,e2433cb4,bf7fdf4c,100,e2433c84,ca35cd40,c86f58a0,e2433cb4) at netbsd:sysctl_dispatch+0xc7
sys___sysctl(ca35cd40,e2433d00,e2433d28,0,c9d40374,ca,0,bb782000,e2433d00,c972e52c) at netbsd:sys___sysctl+0xea
syscall(e2433d48,bb7c00b3,bb7d00ab,bb74001f,bb7f001f,4,bf7fdde4,bf7fddb0,bb7c65bc,bf7fdde0) at netbsd:syscall+0xaa
cpu3: End traceback...
Possibly related, we've noticed an "ESTABLISHED socket leak". These
servers run Apache and often the output of "netstat -f inet" shows
ESTABLISHED connections to port 80 which lsof doesn't. Restarting
Apache will clear them all.
If tcpdrop is used on one of the mystery connections, we get this panic:
panic: kernel diagnostic assertion "cred != NULL" failed: file "/misc/devel/netbsd/6.1/src/sys/kern/kern_auth.c", line 292
cpu3: Begin traceback...
kern_assert(c03e74d4,c03e7515,c03f5ff1,c03f53f0,124,0,e278da40,c01f0c7d,0,ca7ced80) at netbsd:kern_assert+0x23
kauth_cred_getuid(0,ca7ced80,2aba24,0,c03969e3,ca195e54,ca7ced80,c86d7268,e278da54,c033d75c) at netbsd:kauth_cred_getuid+0x43
proc_uidmatch(ca7ced80,0,c86d80a8,e278da90,c01cd59a,ca7ced80,8,0,1a,ca195e54) at netbsd:proc_uidmatch+0x2d
socket_listener_cb(ca7ced80,8,0,1a,ca195e54,c9ee27f4,0,1,0,8) at netbsd:socket_listener_cb+0x3c
kauth_authorize_action_internal(1a,ca195e54,c9ee27f4,0,e278dacc,c01cf3fd,c86d80a8,ca7ced80,8,1a) at netbsd:kauth_authorize_action_internal+0x9a
kauth_authorize_action(c86d80a8,ca7ced80,8,1a,ca195e54,c9ee27f4,0,e278dc14,c030f461,ca7ced80) at netbsd:kauth_authorize_action+0x2f
kauth_authorize_network(ca7ced80,8,1a,ca195e54,c9ee27f4,0,0,e278db2c,c02b017e,2) at netbsd:kauth_authorize_network+0x3d
sysctl_net_inet_tcp_ident(e278dc94,0,0,e278dcb4,bf7fec74,100,e278dc84,ca8fda80,c86f59c0,4) at netbsd:sysctl_net_inet_tcp_ident+0x451
sysctl_dispatch(e278dc84,4,0,e278dcb4,bf7fec74,100,e278dc84,ca8fda80,c86f59c0,e278dcb4) at netbsd:sysctl_dispatch+0xc7
sys___sysctl(ca8fda80,e278dd00,e278dd28,0,c96ed6cc,ca,0,bb78b000,e278dd00,c8c8aa3c) at netbsd:sys___sysctl+0xea
syscall(e278dd48,b3,ab,bf7f001f,806001f,4,0,bf7fe400,bb7c65bc,0) at netbsd:syscall+0xaa
cpu3: End traceback...
>How-To-Repeat:
I don't know how to cause the mystery connections, but they are almost
always present on at least two of our web servers.
Running tcpdrop on the mystery connections will always cause the second
panic.
>Fix:
Unknown
>Release-Note:
>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/48098: panic: kernel diagnostic assertion "cred != NULL": sys/kern/kern_auth.c
Date: Mon, 5 Aug 2013 09:39:51 +0200
I think this needs (for both "do_drop" true and false cases) the same
treatment as kern/47598 to deal with embryonic connections w/o
credentials yet. (c.f. rev 1.5 of
src/sys/secmodel/extensions/secmodel_extensions.c)
Martin
From: Brian Marcotte <marcotte@panix.com>
To: Martin Husemann <martin@duskware.de>
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gnats-bugs@NetBSD.org
Subject: Re: kern/48098: panic: kernel diagnostic assertion "cred != NULL":
sys/kern/kern_auth.c
Date: Thu, 22 Aug 2013 02:24:34 -0400
> I think this needs (for both "do_drop" true and false cases) the same
> treatment as kern/47598 to deal with embryonic connections w/o
> credentials yet. (c.f. rev 1.5 of
> src/sys/secmodel/extensions/secmodel_extensions.c)
If you can give me more to go on, I can probably make the change to
my sources.
Also, I don't know if this is related, but one of the machines which
has had the "cred != NULL" panic has now seen a few of these panics:
panic: kernel diagnostic assertion "(l->l_pflag & LP_INTR) == 0" failed: file "/misc/devel/netbsd/6.1/src/sys/kern/kern_synch.c", line 627
cpu2: Begin traceback...
kern_assert(c03e74d4,c03e7515,c03f9c15,c03f9da4,273,e2be6b4c,c03401b3,e2be6adc,ccc6a690,e2be6af8) at netbsd:kern_assert+0x23
mi_switch(cb926560,e2be6b20,c0203567,c0420ac0,c01eaee7,c86e3402,1,c8776b00,cb6812a0,cb926560) at netbsd:mi_switch+0x76e
sleepq_block(0,0,c03fac69,c0427f8c,c0396965,e2be6b5c,1,c0,cb682f01,cb926560) at netbsd:sleepq_block+0xad
turnstile_block(0,1,c86d6f40,c0427f8c,c0196a55,400,cb6812a0,ccceb5fc,cce693a0,ccc6a690) at netbsd:turnstile_block+0x2af
mutex_vector_enter(c86d6f40,c8776a82,fffffffe,c0396965,ccceb601,e2be6bd4,c03969e3,16e37,0,1fffc000) at netbsd:mutex_vector_enter+0x1c8
tcp_close(ccceb5fc,c0396965,c86eb000,c86eb060,e2be6c24,c020e281,ccceb5fc,e2be6c24,c03969e3,cb926560) at netbsd:tcp_close+0x101
tcp_timer_2msl(ccceb5fc,e2be6c24,c03969e3,cb926560,0,c030b9e0,c86eb004,100,ccceb5fc,e010d214) at netbsd:tcp_timer_2msl+0xe8
callout_softclock(0,0,e2be6c64,52,14,b7bca000,e2be6c94,c02e8b0a,e010d074,40) at netbsd:callout_softclock+0x191
softint_overlay(14,e2be6d00,9,e2be6cc4,c02e8bdf,14,cb926560,cb569564,b7b554b4,e2be6d3c) at netbsd:softint_overlay+0x137
lwp_userret(cb926560,cb926560,e2be6d48,c0425464,e2be6d3c,c02f528a,cb926560,e2be6d00,e2be6d28,c02acc36) at netbsd:lwp_userret+0x238
trap() at netbsd:trap+0x658
--- trap (number 3) ---
baa50c71:
cpu2: End traceback...
I can open a separate ticket if you think it's unrelated.
Thanks.
--
- Brian
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: src/sys/netinet
Date: Fri, 4 Oct 2013 12:20:36 -0400
Module Name: src
Committed By: christos
Date: Fri Oct 4 16:20:35 UTC 2013
Modified Files:
src/sys/netinet: tcp_usrreq.c
Log Message:
PR/48098: Brian Marcotte: Avoid kernel assertion for embryonic sockets that
don't have credentials yet.
XXX: pullup-6
To generate a diff of this commit:
cvs rdiff -u -r1.167 -r1.168 src/sys/netinet/tcp_usrreq.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: [netbsd-6] src/sys/netinet
Date: Sun, 20 Oct 2013 13:29:37 +0000
Module Name: src
Committed By: bouyer
Date: Sun Oct 20 13:29:37 UTC 2013
Modified Files:
src/sys/netinet [netbsd-6]: tcp_usrreq.c
Log Message:
Pull up following revision(s) (requested by spz in ticket #967):
sys/netinet/tcp_usrreq.c: revision 1.168
PR/48098: Brian Marcotte: Avoid kernel assertion for embryonic sockets that
don't have credentials yet.
XXX: pullup-6
To generate a diff of this commit:
cvs rdiff -u -r1.162.2.1 -r1.162.2.2 src/sys/netinet/tcp_usrreq.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: [netbsd-6-1] src/sys/netinet
Date: Sun, 20 Oct 2013 13:29:44 +0000
Module Name: src
Committed By: bouyer
Date: Sun Oct 20 13:29:44 UTC 2013
Modified Files:
src/sys/netinet [netbsd-6-1]: tcp_usrreq.c
Log Message:
Pull up following revision(s) (requested by spz in ticket #967):
sys/netinet/tcp_usrreq.c: revision 1.168
PR/48098: Brian Marcotte: Avoid kernel assertion for embryonic sockets that
don't have credentials yet.
XXX: pullup-6
To generate a diff of this commit:
cvs rdiff -u -r1.162.2.1 -r1.162.2.1.6.1 src/sys/netinet/tcp_usrreq.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: [netbsd-6-0] src/sys/netinet
Date: Sun, 20 Oct 2013 13:29:49 +0000
Module Name: src
Committed By: bouyer
Date: Sun Oct 20 13:29:48 UTC 2013
Modified Files:
src/sys/netinet [netbsd-6-0]: tcp_usrreq.c
Log Message:
Pull up following revision(s) (requested by spz in ticket #967):
sys/netinet/tcp_usrreq.c: revision 1.168
PR/48098: Brian Marcotte: Avoid kernel assertion for embryonic sockets that
don't have credentials yet.
XXX: pullup-6
To generate a diff of this commit:
cvs rdiff -u -r1.162.2.1 -r1.162.2.1.4.1 src/sys/netinet/tcp_usrreq.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: bouyer@NetBSD.org
State-Changed-When: Sun, 20 Oct 2013 21:10:45 +0000
State-Changed-Why:
A fix has been commited and pulled up to all netbsd-6 branches.
Is the problem fixed for you ?
From: Brian Marcotte <marcotte@panix.com>
To: gnats-bugs@NetBSD.org
Cc: Brian Marcotte <marcotte@panix.com>
Subject: Re: kern/48098 (panic: kernel diagnostic assertion "cred != NULL":
sys/kern/kern_auth.c)
Date: Mon, 21 Oct 2013 22:41:00 -0400
> Synopsis: panic: kernel diagnostic assertion "cred != NULL": ... /kern_auth.c
> A fix has been commited and pulled up to all netbsd-6 branches.
Sorry for the delay, due to illness, it took me a over a week to install
the patch on my production servers. I've now had it installed for a week.
At some point after I filed the initial bug report, I stopped getting
these panics on a regular basis.
However, I did do a "tcpdrop" on one of the ESTABLISHED connections
that have no corresponding entry in lsof/fstat and got the second panic
described in the ticket.
So, only time will tell if the initial problem is fixed, but there is
still that weird issue with ESTABLISHED connections that I can't even
work around with tcpdrop.
Thanks.
--
- Brian
State-Changed-From-To: feedback->open
State-Changed-By: bouyer@NetBSD.org
State-Changed-When: Tue, 22 Oct 2013 07:43:41 +0000
State-Changed-Why:
The second panic, caused by tcpdrop, is still here.
From: "S.P.Zeidler" <spz@NetBSD.org>
To: Brian Marcotte <marcotte@panix.com>
Cc: gnats-bugs@gnats.NetBSD.org
Subject: Re: kern/48098: panic: kernel diagnostic assertion
Date: Sun, 27 Oct 2013 18:00:18 +0000
--fUYQa+Pmc3FrFX/N
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Hello Brian,
could you please test if the attached patch (kudos mlelstv@NetBSD.org)
helps with the tcpdrop issue? it's only lightly tested so maybe not
on your most precious server. :)
For the reason there are stuck connections in the first place:
when you have some, could you please check if an apache thread is
stuck either in soaccept or kauth_cred_dup, from do_sys_accept?
regards,
spz
--fUYQa+Pmc3FrFX/N
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="48098.patch-candidate"
Index: sys/netinet/tcp_usrreq.c
===================================================================
RCS file: /cvsroot/src/sys/netinet/tcp_usrreq.c,v
retrieving revision 1.168
diff -u -r1.168 tcp_usrreq.c
--- sys/netinet/tcp_usrreq.c 4 Oct 2013 16:20:35 -0000 1.168
+++ sys/netinet/tcp_usrreq.c 27 Oct 2013 17:36:13 -0000
@@ -1199,12 +1199,16 @@
struct tcpcb *tp;
int error;
- if (inp == NULL || (tp = intotcpcb(inp)) == NULL ||
- (inp->inp_socket->so_options & SO_ACCEPTCONN) != 0)
+ solock(sockp);
+ if ((tp = intotcpcb(inp)) == NULL ||
+ (sockp->so_options & SO_ACCEPTCONN) != 0) {
+ sounlock(sockp);
return ESRCH;
+ }
error = kauth_authorize_network(l->l_cred, KAUTH_NETWORK_SOCKET,
- KAUTH_REQ_NETWORK_SOCKET_DROP, inp->inp_socket, tp, NULL);
+ KAUTH_REQ_NETWORK_SOCKET_DROP, sockp, tp, NULL);
+ sounlock(sockp);
if (error)
return (error);
@@ -1234,12 +1238,16 @@
struct tcpcb *tp;
int error;
- if (in6p == NULL || (tp = in6totcpcb(in6p)) == NULL ||
- (in6p->in6p_socket->so_options & SO_ACCEPTCONN) != 0)
+ solock(sockp);
+ if ((tp = in6totcpcb(in6p)) == NULL ||
+ (sockp->so_options & SO_ACCEPTCONN) != 0) {
+ sounlock(sockp);
return ESRCH;
+ }
error = kauth_authorize_network(l->l_cred, KAUTH_NETWORK_SOCKET,
- KAUTH_REQ_NETWORK_SOCKET_DROP, in6p->in6p_socket, tp, NULL);
+ KAUTH_REQ_NETWORK_SOCKET_DROP, sockp, tp, NULL);
+ sounlock(sockp);
if (error)
return (error);
--fUYQa+Pmc3FrFX/N--
From: Brian Marcotte <marcotte@panix.com>
To: "S.P.Zeidler" <spz@NetBSD.org>
Cc: gnats-bugs@gnats.NetBSD.org, Brian Marcotte <marcotte@panix.com>
Subject: Re: kern/48098: panic: kernel diagnostic assertion
Date: Mon, 28 Oct 2013 08:20:30 -0400
> could you please test if the attached patch (kudos mlelstv@NetBSD.org)
> helps with the tcpdrop issue? it's only lightly tested so maybe not
> on your most precious server. :)
That patch seems to make things worse. With it, I get this panic using
tcpdrop on connections in FIN_WAIT_1 (possibly others):
Mutex error: mutex_vector_enter: locking against myself
lock address : 0x00000000c104ff40
current cpu : 0
current lwp : 0x00000000c24fa540
owner field : 0x00000000c24fa540 wait/spin: 0/0
panic: lock error
cpu0: Begin traceback...
panic(c0408709,c03f8468,c03d8841,c03f8437,c104ff40,0,c24fa540,c0428ae0,c0428ae0,d8651a9c) at netbsd:panic+0x18
lockdebug_abort(c104ff40,c0428ae0,c03d8841,c03f8437,c104ff40,c24fa540,d8651acc,c01eb13b,c208fa00,ffffffff) at netbsd:lockdebug_abort+0x2f
mutex_abort(c208fa00,ffffffff,c6f53133,c257921c,c24d3744,bb018588,c12feab0,c104ff40,c24f5174,0) at netbsd:mutex_abort+0x32
mutex_vector_enter(c104ff40,3331f5c6,6fcb,ed0854a6,bb01,0,0,d8651b2c,c24d3744,2) at netbsd:mutex_vector_enter+0x33b
sysctl_net_inet_tcp_ident(d8651c94,0,0,d8651cb4,bf7fec70,100,d8651c84,c24fa540,c106b9c0,4) at netbsd:sysctl_net_inet_tcp_ident+0x383
sysctl_dispatch(d8651c84,4,0,d8651cb4,bf7fec70,100,d8651c84,c24fa540,c106b9c0,d8651cb4) at netbsd:sysctl_dispatch+0xc7
sys___sysctl(c24fa540,d8651d00,d8651d28,0,c132bbd0,ca,0,bb78b000,d8651d00,c256ba58) at netbsd:sys___sysctl+0xea
syscall(d8651d48,b3,ab,bf7f001f,806001f,4,0,bf7fe3fc,bb7c65bc,0) at netbsd:syscall+0xaa
cpu0: End traceback...
> For the reason there are stuck connections in the first place:
> when you have some, could you please check if an apache thread is
> stuck either in soaccept or kauth_cred_dup, from do_sys_accept?
I don't see those in the WCHAN in our "ps" logs, so that sounds like
something to do in ddb. Are you asking that I do this on all the apache
processes?
bt /t 0t[pid]
I managed to do this when there were stray ESTABLISHED connections, but
I don't see any of what you asked for. I only see these:
sleepq_block(...) at netbsd:sleepq_block+0xad
sel_do_scan(...) at netbsd:sel_do_scan+0x4a8
selcommon(...) at netbsd:selcommon+0x1f9
sys___select50(...) at netbsd:sys___select50+0x77
syscall(...) at netbsd:syscall+0xaa
sleepq_block(...) at netbsd:sleepq_block+0xad
cv_wait_sig(...) at netbsd:cv_wait_sig+0x103
lf_advlock(...) at netbsd:lf_advlock+0x44a
ufs_advlock(...) at netbsd:ufs_advlock+0x36
VOP_ADVLOCK(...) at netbsd:VOP_ADVLOCK+0x41
sys_flock(...) at netbsd:sys_flock+0xe2
syscall(...) at netbsd:syscall+0xaa
sleepq_block(...) at netbsd:sleepq_block+0xad
cv_timedwait_sig(...) at netbsd:cv_timedwait_sig+0x101
kevent1(...) at netbsd:kevent1+0x45a
sys___kevent50(...) at netbsd:sys___kevent50+0x45
syscall(...) at netbsd:syscall+0xaa
This is the normal state of things. The apache processes are normally
waiting in "lockf","select", or "kqueue".
Interestingly, for the DEBUG/DIAGNOSTIC/LOCKDEBUG kernels, if I do that
in ddb, and then do "cont", I get this panic (even without your patch):
panic: SPL NOT LOWERED ON TRAP EXIT
cpu0: Begin traceback...
panic(c0102c0f,ca010011,c0110031,1d6a0011,28f0011,c0b5fe08,0,ca015f38,c066d005,2b) at netbsd:panic+0x18
alltraps(c01eb01d,c0431702,5,1,6,ca015f94,c039cd28,c0ad7668,c066d005,1) at netbsd:alltraps+0x17e
xencons_tty_input(c0ad7668,c066d005,1,6,ca015fa8,ca015f94,c01e5164,c054f800,c01eb01d,c043496a) at netbsd:xencons_tty_input+0xb8
xencons_handler(c0ad7668,2,c0adc128,ca015fe8,c013ec6b,c0adc128,ca174ca8,ca015fec,c016fa87,a) at netbsd:xencons_handler+0x78
intr_biglock_wrapper(c0adc128,ca174ca8,ca015fec,c016fa87,a,2,0,0,c0434968,40) at netbsd:intr_biglock_wrapper+0x1f
evtchn_do_event(2,ca174ca8,ca174c54,0,0,0,0,0,0,0) at netbsd:evtchn_do_event+0x16b
--- switch to interrupt stack ---
call_evtchn_do_event(ca174ca8,0,c0420011,ca170031,c0200011,11,c0b46d40,c04214c0,ca174d04,1) at netbsd:call_evtchn_do_event+0x1e
hypervisor_callback(c0b48d20,0,0,ca174da0,c0b48d20,c0b48d20,c01e0cf0,c0b48d20,0,c01000a1) at netbsd:hypervisor_callback+0x64
idle_loop(c0b48d20,67c000,c057b200,0,c010006d,0,0,0,0,0) at netbsd:idle_loop+0x17c
cpu0: End traceback...
That doesn't prevent me from getting the trace on the processes, so it's
not a big problem.
Thanks.
--
- Brian
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48098: panic: kernel diagnostic assertion "cred != NULL":
sys/kern/kern_auth.c
Date: Sat, 2 Nov 2013 19:38:03 +0000
sent to "gnats" instead of "gnats-bugs"
------
From: Michael van Elst <mlelstv@serpens.de>
To: gnats@netbsd.org
Subject: Re: kern/48098: panic: kernel diagnostic assertion "cred != NULL":
sys/kern/kern_auth.c
Date: Mon, 28 Oct 2013 22:37:32 +0100
Our network code adds new connections to the connection table
in an interrupt and references a socket from this table. The
socket is also added to the accept queue of the listening socket.
At this point the socket has no credentials, but tcpdrop can
find it in the connection table and crash the system by referencing
a NULL pointer.
Index: uipc_socket.c
===================================================================
RCS file: /cvsroot/src/sys/kern/uipc_socket.c,v
retrieving revision 1.219
diff -u -r1.219 uipc_socket.c
--- uipc_socket.c 17 Oct 2013 20:57:06 -0000 1.219
+++ uipc_socket.c 28 Oct 2013 21:26:12 -0000
@@ -416,7 +416,7 @@
/* Normal users can only drop their own connections. */
struct socket *so = (struct socket *)arg1;
- if (proc_uidmatch(cred, so->so_cred) == 0)
+ if (so->so_cred != NULL && proc_uidmatch(cred, so->so_cred) == 0)
result = KAUTH_RESULT_ALLOW;
break;
This patch should prevent tcpdrop from crashing the system. Since the
not accepted socket has no credentials, the connection can only be
dropped by the superuser.
Greetings,
--
Michael van Elst
Internet: mlelstv@serpens.de
"A potential Snark may lurk in every tree."
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: src/sys/kern
Date: Sat, 2 Nov 2013 16:09:33 -0400
Module Name: src
Committed By: christos
Date: Sat Nov 2 20:09:33 UTC 2013
Modified Files:
src/sys/kern: uipc_socket.c
Log Message:
PR/48098: Brian Marcotte: panic: kernel diagnostic assertion "cred != NULL":
Fix from Michael van Elst, tcpdrop crashes kernel on ebryonic connections.
To generate a diff of this commit:
cvs rdiff -u -r1.219 -r1.220 src/sys/kern/uipc_socket.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 03 Nov 2013 00:01:37 +0000
State-Changed-Why:
Does that fix help?
From: Brian Marcotte <marcotte@panix.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/48098 (panic: kernel diagnostic assertion "cred != NULL":
sys/kern/kern_auth.c)
Date: Wed, 6 Nov 2013 20:02:45 -0500
> Modified Files:
> src/sys/kern: uipc_socket.c
Okay, I can now drop these connections without causing a panic.
Within the next week, I need to get this onto one of our busier machines.
Over there, apache occasionally seems to stop accepting SOME connections
when there are a large number of these stale established connections.
With this patch, I hope to work around that problem by automatically
dropping them. Right now, I must restart apache (which clears all the
stuck established connections).
Any guess why these pile up? I've only seen this behavior on some of our
web servers. It does not happen on our web servers which are protected
by a Squid box, nor the Squid box itself. It doesn't happen on our mail
servers which also handle lots of connections.
Thanks.
--
- brian
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: [netbsd-6] src/sys/kern
Date: Mon, 25 Nov 2013 08:26:33 +0000
Module Name: src
Committed By: bouyer
Date: Mon Nov 25 08:26:33 UTC 2013
Modified Files:
src/sys/kern [netbsd-6]: uipc_socket.c
Log Message:
Pull up following revision(s) (requested by spz in ticket #988):
sys/kern/uipc_socket.c: revision 1.220
PR/48098: Brian Marcotte: panic: kernel diagnostic assertion "cred != NULL":
Fix from Michael van Elst, tcpdrop crashes kernel on ebryonic connections.
To generate a diff of this commit:
cvs rdiff -u -r1.209.2.3 -r1.209.2.4 src/sys/kern/uipc_socket.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: [netbsd-6-0] src/sys/kern
Date: Mon, 25 Nov 2013 08:27:01 +0000
Module Name: src
Committed By: bouyer
Date: Mon Nov 25 08:27:01 UTC 2013
Modified Files:
src/sys/kern [netbsd-6-0]: uipc_socket.c
Log Message:
Pull up following revision(s) (requested by spz in ticket #988):
sys/kern/uipc_socket.c: revision 1.220
PR/48098: Brian Marcotte: panic: kernel diagnostic assertion "cred != NULL":
Fix from Michael van Elst, tcpdrop crashes kernel on ebryonic connections.
To generate a diff of this commit:
cvs rdiff -u -r1.209.2.1.4.1 -r1.209.2.1.4.2 src/sys/kern/uipc_socket.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48098 CVS commit: [netbsd-6-1] src/sys/kern
Date: Mon, 25 Nov 2013 08:27:07 +0000
Module Name: src
Committed By: bouyer
Date: Mon Nov 25 08:27:06 UTC 2013
Modified Files:
src/sys/kern [netbsd-6-1]: uipc_socket.c
Log Message:
Pull up following revision(s) (requested by spz in ticket #988):
sys/kern/uipc_socket.c: revision 1.220
PR/48098: Brian Marcotte: panic: kernel diagnostic assertion "cred != NULL":
Fix from Michael van Elst, tcpdrop crashes kernel on ebryonic connections.
To generate a diff of this commit:
cvs rdiff -u -r1.209.2.2.2.1 -r1.209.2.2.2.2 src/sys/kern/uipc_socket.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 01 May 2016 05:18:07 +0000
State-Changed-Why:
2.5-year feedback timeout.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home