NetBSD Problem Report #55848

From paul@whooppee.com  Mon Dec  7 16:43:42 2020
Return-Path: <paul@whooppee.com>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id A72361A9217
	for <gnats-bugs@gnats.NetBSD.org>; Mon,  7 Dec 2020 16:43:42 +0000 (UTC)
Message-Id: <20201207164337.C595530F2C4@speedy.whooppee.com>
Date: Mon,  7 Dec 2020 08:43:37 -0800 (PST)
From: paul@whooppee.com
Reply-To: paul@whooppee.com
To: gnats-bugs@NetBSD.org
Subject: amd64 9/99.76 panic in audio(4)
X-Send-Pr-Version: 3.95

>Number:         55848
>Category:       kern
>Synopsis:       amd64 9/99.76 panic in audio(4)
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    isaki
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Dec 07 16:45:00 +0000 2020
>Closed-Date:    Mon Dec 21 03:58:27 +0000 2020
>Last-Modified:  Mon Dec 21 03:58:27 +0000 2020
>Originator:     Paul Goyette
>Release:        NetBSD 9.99.76
>Organization:
+--------------------+--------------------------+-----------------------+
| Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
| (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
+--------------------+--------------------------+-----------------------+
>Environment:


System: NetBSD speedy.whooppee.com 9.99.76 NetBSD 9.99.76 (SPEEDY 2020-12-04 23:39:34 UTC) #0: Sat Dec 5 07:30:37 UTC 2020 paul@speedy.whooppee.com:/build/netbsd-local/obj/amd64/sys/arch/amd64/compile/SPEEDY amd64
Architecture: x86_64
Machine: amd64
>Description:
	With amd64 9.99.76, if I create a shortage of kernel file
	table entries the audio driver panics during open(2).
	The symptom starts with thousands of the following error
	message within just a few seconds

[ 37907.1552432] file: table is full - increase kern.maxfiles or MAXFILES
[ 37907.1552432] file: table is full - increase kern.maxfiles or MAXFILES
[ 37907.1552432] file: table is full - increase kern.maxfiles or MAXFILES
...  (6,776 identical lines snipped)
[ 37928.6031590] file: table is full - increase kern.maxfiles or MAXFILES
[ 37928.6231664] file: table is full - increase kern.maxfiles or MAXFILES
[ 37928.6231664] file: table is full - increase kern.maxfiles or MAXFILES

	Then the system panics:

[ 37928.6231664] panic: kernel diagnostic assertion "sc->sc_rbusy == false" fail
ed: file "/build/netbsd-local/src_ro/sys/dev/audio/audio.c", line 5587
[ 37928.6231664] cpu1: Begin traceback...
[ 37928.6231664] vpanic() at netbsd:vpanic+0x156
[ 37928.6231664] __x86_indirect_thunk_rax() at netbsd:__x86_indirect_thunk_rax
[ 37928.6231664] audio_rmixer_start() at audio:audio_rmixer_start+0xe7
[ 37928.6231664] audio_open.isra.0() at audio:audio_open.isra.0+0x2ad
[ 37928.6231664] audioopen() at audio:audioopen+0x18f
[ 37928.6231664] spec_open() at netbsd:spec_open+0x176
[ 37928.6231664] VOP_OPEN() at netbsd:VOP_OPEN+0x3c
[ 37928.6231664] vn_open() at netbsd:vn_open+0x130
[ 37928.6231664] do_open() at netbsd:do_open+0x119
[ 37928.6231664] do_sys_openat() at netbsd:do_sys_openat+0x74 
[ 37928.6231664] sys_open() at netbsd:sys_open+0x24
[ 37928.6231664] syscall() at netbsd:syscall+0x23e
[ 37928.6231664] --- syscall (number 5) ---
[ 37928.6231664] netbsd:syscall+0x23e: 
[ 37928.6231664] cpu1: End traceback...
[ 37928.6231664] fatal breakpoint trap in supervisor mode
[ 37928.6231664] trap type 1 code 0 rip 0xffffffff8021f415 cs 0x8 rflags 0x202 c
r2 0x7c954ff98000 ilevel 0 rsp 0xffff8b892c18ca50
[ 37928.6231664] curlwp 0xffff81b1ff052b00 pid 1537.1538 lowest kstack 0xffff8b8
92c1882c0
Stopped in pid 1537.1538 (pulseaudio) at        netbsd:breakpoint+0x5:  leave
breakpoint() at netbsd:breakpoint+0x5
>How-To-Repeat:
	It's not 100% repeatable, but I have gotten this crash twice
	under the following conditions:

	1. Use sysctl to reduce kern.maxfiles 20000 --> 5120

	2. Run firefox, open up a lot of tabs, and switch to a tab
	   which has an audio stream

(Note that the reduction of kern.maxfiles was "an accident"!  It was
a left-over entry in /etc/sysctl.conf which was added a long time ago
when firefox used to exceed the default value of ~3k.  Recent changes
increased the default value - to 20000 on my system - so the entry in
sysctl.conf was a significant reduction/restriction!)

>Fix:
	No fix available


>Release-Note:

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Mon, 7 Dec 2020 17:48:01 +0100

 Can you please (with a single tab using audio) check how many files
 firefox uses in your setup and (e.g. with ktrace) check if it constantly
 opens or close some of them?

 What audio backend is FF using for you?

 Martin

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Mon, 7 Dec 2020 09:36:47 -0800 (PST)

 On Mon, 7 Dec 2020, Martin Husemann wrote:

 > Can you please (with a single tab using audio) check how many files
 > firefox uses in your setup and (e.g. with ktrace) check if it constantly
 > opens or close some of them?

 The specific sub-process-with-audio has

  	# ls /proc/1841/fd | wc -w
  	165
  	#

 > What audio backend is FF using for you?

 I have no  idea - how do I check?



 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Mon, 7 Dec 2020 10:05:39 -0800 (PST)

 Looking at the code and traceback, it would seem that firefox (or
 something) is attempting to open the record-mixer without ensuring
 that sc_rbusy is unset.


 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: paul@whooppee.com
Cc: gnats-bugs@netbsd.org, tsutsui@ceres.dti.ne.jp
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Tue, 8 Dec 2020 03:25:02 +0900

 > > What audio backend is FF using for you?
 > 
 > I have no  idea - how do I check?

 about:support -> media

 https://support.mozilla.org/kb/use-troubleshooting-information-page-fix-firefox

 ---
 Izumi Tsutsui

From: Paul Goyette <paul@whooppee.com>
To: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Mon, 7 Dec 2020 10:30:22 -0800 (PST)

 On Tue, 8 Dec 2020, Izumi Tsutsui wrote:

 >>> What audio backend is FF using for you?
 >>
 >> I have no  idea - how do I check?
 >
 > about:support -> media

 Ah!

 Output Devices
 Name 	Group 	Vendor 	State 	Preferred 	Format 	Channels 
 Rate 	Latency
 /dev/audio			Enabled	All	default: S16LE, support: 
 S16LE S16BE F32LE F32BE	2	default: 44100, support: 1 - 384000 
 0 - 0


 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

Responsible-Changed-From-To: kern-bug-people->isaki
Responsible-Changed-By: isaki@NetBSD.org
Responsible-Changed-When: Tue, 08 Dec 2020 06:09:43 +0000
Responsible-Changed-Why:
My bug.


From: "Tetsuya Isaki" <isaki@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55848 CVS commit: src/sys/dev/audio
Date: Wed, 9 Dec 2020 04:24:08 +0000

 Module Name:	src
 Committed By:	isaki
 Date:		Wed Dec  9 04:24:08 UTC 2020

 Modified Files:
 	src/sys/dev/audio: audio.c

 Log Message:
 Fix that audio_open() didn't halt the recording mixer correctly
 if fd_allocfile() failed, since rev 1.65.
 Will fix PR kern/55848.


 To generate a diff of this commit:
 cvs rdiff -u -r1.79 -r1.80 src/sys/dev/audio/audio.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: isaki@NetBSD.org
State-Changed-When: Wed, 09 Dec 2020 04:46:27 +0000
State-Changed-Why:


From: Tetsuya Isaki <isaki@pastel-flower.jp>
To: Paul Goyette <paul@whooppee.com>
Cc: gnats-bugs@netbsd.org,
	kern-bug-people@netbsd.org,
	gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Wed, 09 Dec 2020 13:45:27 +0900

 At Mon, 7 Dec 2020 10:05:39 -0800 (PST),
 Paul Goyette wrote:
 > Looking at the code and traceback, it would seem that firefox (or
 > something) is attempting to open the record-mixer without ensuring
 > that sc_rbusy is unset.

 I have just committed the fix.  In fact I didn't use firefox to
 reproduce, but I simulated and could confirm the same panic.

 Could you try audio.c rev 1.80 (or later)?

 Thanks,
 ---
 Tetsuya Isaki <isaki@pastel-flower.jp / isaki@NetBSD.org>

From: Paul Goyette <paul@whooppee.com>
To: Tetsuya Isaki <isaki@pastel-flower.jp>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Wed, 9 Dec 2020 05:28:15 -0800 (PST)

 On Wed, 9 Dec 2020, Tetsuya Isaki wrote:

 > At Mon, 7 Dec 2020 10:05:39 -0800 (PST),
 > Paul Goyette wrote:
 >> Looking at the code and traceback, it would seem that firefox (or
 >> something) is attempting to open the record-mixer without ensuring
 >> that sc_rbusy is unset.
 >
 > I have just committed the fix.  In fact I didn't use firefox to
 > reproduce, but I simulated and could confirm the same panic.
 >
 > Could you try audio.c rev 1.80 (or later)?

 Yeah, I plan to update again on the weekend.


 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: Paul Goyette <paul@whooppee.com>
To: Tetsuya Isaki <isaki@pastel-flower.jp>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Wed, 9 Dec 2020 19:37:30 -0800 (PST)

 On Wed, 9 Dec 2020, Tetsuya Isaki wrote:

 > I have just committed the fix.  In fact I didn't use firefox to
 > reproduce, but I simulated and could confirm the same panic.
 > 
 > Could you try audio.c rev 1.80 (or later)?

 I just updated to today's -current (9.99.77) and everything looks good.

 Please close the PR (or submit pullups if appropriate).


 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: Tetsuya Isaki <isaki@pastel-flower.jp>
To: gnats-bugs@netbsd.org, paul@whooppee.com
Cc: gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: kern/55848: amd64 9/99.76 panic in audio(4)
Date: Fri, 11 Dec 2020 11:25:32 +0900

 At Thu, 10 Dec 2020 03:40:01 +0000 (UTC),
 Paul Goyette wrote:
 >  > Could you try audio.c rev 1.80 (or later)?
 >  
 >  I just updated to today's -current (9.99.77) and everything looks good.
 >  
 >  Please close the PR (or submit pullups if appropriate).

 Thank you.  I will pullup it in next week.
 ---
 Tetsuya Isaki <isaki@pastel-flower.jp / isaki@NetBSD.org>

State-Changed-From-To: feedback->pending-pullups
State-Changed-By: isaki@NetBSD.org
State-Changed-When: Sat, 19 Dec 2020 04:13:32 +0000
State-Changed-Why:
[pullup-9 #1156]


From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55848 CVS commit: [netbsd-9] src/sys/dev/audio
Date: Sat, 19 Dec 2020 13:48:27 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Sat Dec 19 13:48:27 UTC 2020

 Modified Files:
 	src/sys/dev/audio [netbsd-9]: audio.c

 Log Message:
 Pull up following revision(s) (requested by isaki in ticket #1156):

 	sys/dev/audio/audio.c: revision 1.80
 	sys/dev/audio/audio.c: revision 1.81

 Fix that audio_open() didn't halt the recording mixer correctly
 if fd_allocfile() failed, since rev 1.65.

 Will fix PR kern/55848.

  -

 Rewrite error handling on audio_open().
 This also fixes a few resource leaks on error case.


 To generate a diff of this commit:
 cvs rdiff -u -r1.28.2.16 -r1.28.2.17 src/sys/dev/audio/audio.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: isaki@NetBSD.org
State-Changed-When: Mon, 21 Dec 2020 03:58:27 +0000
State-Changed-Why:
Pullup was applied.


>Unformatted:

Home	PR Database Search