NetBSD Problem Report #50539

From tsutsui@ceres.dti.ne.jp  Fri Dec 11 18:00:00 2015
Return-Path: <tsutsui@ceres.dti.ne.jp>
Received: from mail.netbsd.org (mail.NetBSD.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 35EAAA5807
	for <gnats-bugs@gnats.NetBSD.org>; Fri, 11 Dec 2015 18:00:00 +0000 (UTC)
Message-Id: <201512111759.tBBHxsKX004342@mirage.localdomain>
Date: Sat, 12 Dec 2015 02:59:54 +0900 (JST)
From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Reply-To: tsutsui@ceres.dti.ne.jp
To: gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: pulseaudio consumes too much /var/shm for shm_open(3)
X-Send-Pr-Version: 3.95

>Number:         50539
>Category:       pkg
>Synopsis:       pulseaudio consumes too much /var/shm for shm_open(3)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    ryoon
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Fri Dec 11 18:05:00 +0000 2015
>Closed-Date:    
>Last-Modified:  Tue Dec 29 01:39:53 +0000 2015
>Originator:     Izumi Tsutsui
>Release:        NetBSD 7.0
>Organization:
>Environment:
System: NetBSD/i386 7.0 (GENERIC.201509250726Z) i386
Architecture: i386
Machine: i386
>Description:
It looks pulseaudio creates 64MB shm files in /var/shm via shm_open(3)
per each pulseaudio client.

On the other hand, the default NetBSD sysinst installation prepares
the following fstab(5) entry:
>> tmpfs /var/shm tmpfs rw,-m1777,-sram%25

This means:
 - On 1GB RAM machines, /var/shm will have only(?) ~256 MB
 - Pulseaudio can handle only three clients with ~256 MB /var/shm
   and it will complain about shm failure like:

>>  Dec  6 14:05:36 mirage pulseaudio[29034]: [(null)] shm.c: ftruncate() failed: No space left on device

 - Even if the machine have enough tmpfs, it seems multiple 64 MB
   shm files on tmpfs could cause extreme high CPU load,
   as mentioned in PR/50306
http://gnats.netbsd.org/50306
>> html5 sound output in firefox with the pulseaudio default eats
>> a lot of CPU cycles and very often results in "audiostream underrun"
>> and stuttering of the sound on my oldish Asus P4B533 with
>> Pentium 4 @ 2.5GHz

Note if /var/shm is not mounted as tmpfs (that could happen on users
who didn't use sysinst on upgrade), pulseaudio also complains:

>> shm.c: shm_open() failed: Not supported

>How-To-Repeat:
Install pulseaudio and clients (like firefox html5 player, onscripter,
mplayer etc.) via pkgsrc and execute them?

>Fix:
In pulseaudio-7.1/src/pulsecore/memblock.c there is the following comment:
---
/* We can allocate 64*1024*1024 bytes at maximum. That's 64MB. Please
 * note that the footprint is usually much smaller, since the data is
 * stored in SHM and our OS does not commit the memory before we use
 * it for the first time. */
---

It looks the authors assume "our OS" is always Linux, but unfortunately
it does not seem true on shm_open(3) implementation of NetBSD 7.x.

I've tried shm setting to 8MB on NetBSD/i386 7.0 and
firefox html5 video is played smoothly without particular problem
on my Atom N455 + 1GB RAM (i.e. with 256MB tmpfs) netbook.

---
Index: MESSAGE.NetBSD
===================================================================
RCS file: MESSAGE.NetBSD
diff -N MESSAGE.NetBSD
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ MESSAGE.NetBSD	11 Dec 2015 17:45:41 -0000
@@ -0,0 +1,11 @@
+===========================================================================
+$NetBSD$
+
+If you are running NetBSD 7.0 or later please make sure that /var/shm is
+mounted as tmpfs for pulseaudio to use shm_open(2):
+
+This can be automated by adding the following to your /etc/fstab:
+
+tmpfs /var/shm tmpfs rw,-m1777,-sram%25
+
+===========================================================================
Index: distinfo
===================================================================
RCS file: /cvsroot/pkgsrc/audio/pulseaudio/distinfo,v
retrieving revision 1.55
diff -u -p -d -r1.55 distinfo
--- distinfo	10 Dec 2015 10:54:50 -0000	1.55
+++ distinfo	11 Dec 2015 17:45:41 -0000
@@ -13,6 +13,7 @@ SHA1 (patch-src_modules_module-detect.c)
 SHA1 (patch-src_modules_module-solaris.c) = 03ea8912da4bb76294ca23271e47d40a8c9a61a7
 SHA1 (patch-src_modules_oss_module-oss.c) = 399ac178ae832619253ce8dd985edbed23db86e7
 SHA1 (patch-src_pulsecore_core-util.c) = 186c805ac4083446afe3ce8db5c6157da92871be
+SHA1 (patch-src_pulsecore_memblock.c) = 9ceb5d7e489a730875c1c4a5bd9676e5de5cea20
 SHA1 (patch-src_pulsecore_mix__neon.c) = 6f6d33d38024d65045d637d48276e1ba92b81342
 SHA1 (patch-src_pulsecore_sample-util.h) = b6bd83cfdc1c337453d9a728f07205a2cf0af831
 SHA1 (patch-src_pulsecore_svolume__mmx.c) = c34d153e3bfdb812eb7bd70fa330a9ec674c2dc2
Index: patches/patch-src_pulsecore_memblock.c
===================================================================
RCS file: patches/patch-src_pulsecore_memblock.c
diff -N patches/patch-src_pulsecore_memblock.c
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ patches/patch-src_pulsecore_memblock.c	11 Dec 2015 17:45:41 -0000
@@ -0,0 +1,22 @@
+$NetBSD$
+
+Restrict PA_MEMPOLL_SLOT_SIZE to 8MB on NetBSD to avoid shortage of
+/var/shm which is mount as tmpfs with 25% of the physical memory.
+
+--- src/pulsecore/memblock.c.orig	2015-02-12 14:10:35.000000000 +0000
++++ src/pulsecore/memblock.c
+@@ -55,7 +55,14 @@
+  * stored in SHM and our OS does not commit the memory before we use
+  * it for the first time. */
+ #define PA_MEMPOOL_SLOTS_MAX 1024
++#if defined(__NetBSD__)
++/* On NetBSD (i.e. non-"our OS" system), shm_open(3) uses /var/shm
++ * which is usually mounted as tmpfs and it's a bit pain to consume
++ * 64MB for each client (i.e. >256MB tmpfs even only for four clients). */
++#define PA_MEMPOOL_SLOT_SIZE (8*1024)
++#else
+ #define PA_MEMPOOL_SLOT_SIZE (64*1024)
++#endif
+ 
+ #define PA_MEMEXPORT_SLOTS_MAX 128
+ 

---

I know this patch should be submitted to upstream first,
but I'm so lazy and use pulseaudio only via pkgsrc :-p

---

>Release-Note:

>Audit-Trail:
From: Kamil Rytarowski <n54@gmx.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/50539: pulseaudio consumes too much /var/shm for shm_open(3)
Date: Sat, 12 Dec 2015 00:38:30 +0100

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA256

 On 11.12.2015 19:05, Izumi Tsutsui wrote:
 > ---
 > 
 > I know this patch should be submitted to upstream first, but I'm so
 > lazy and use pulseaudio only via pkgsrc :-p
 > 
 > ---
 > 

 Thank you for your work!

 I'm working with upstream to streamline the NetBSD support in
 PulseAudio. I will go with this to upstream developers to research a
 solution.

 At first look I think it should be configurable from a config file and
 just tuned by user. Perhaps with a MESSAGE warning.

 As I'm heading to get next version more correct for us, I will leave
 the decision what to do in pkgsrc for others.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2

 iQIcBAEBCAAGBQJWa151AAoJEEuzCOmwLnZsw34QAMCkCkzmD26WFGltpJLk3pH2
 7488UGLk8IroRzpPe2mkNMGUfmrTlVn6Sc6Zk/ur569CE2g73TYhomt6KHbKOiXe
 MHOX548lsqCodrM+FA0w46BgTqGk96f/qLXuE6NkneV4GhBTFXDYSJt/WKuNn+Ay
 m3orvI1xTO2ykWOfMx8xyUAcAWU+GTCzn5CuxPl5nwwvrF5dLKj3g+EAlbr3U7+Y
 h6mwuoe8nB5ItYbpe8Y4y7R8e+8U+n+T5b8DA0hGnZcfbBCVy85aomMiyzEgUHWG
 JXmjomh5iof4UUsPEkiuGZem8gM5915XLcM/0yy8K4HHMqBssy9293wcGgDp8ZlP
 g3h44C3vzTapl2cQ8jLh6I1JjcLUsjwg062o82d/HmBCeeQAbnO8Xtj1iS2RJoSx
 w/7zRUUhiE7VE+gOXDI0M3R6t+BPex+qPrYiDHAChCoukAeaPRXMmGFK4mNfdq34
 o10udl9xofqXP5BlSGW3pgxRFoev8OKkhPolDgh0N+PRyL/ISfc2TBG0/bmySxkA
 Cr7sYBzLYEjX0o2OezBG5hJKQpuU+BPXXnHWXq1Ez1HkaYqfgrEReyczOk0GeMm9
 plqHTfjRhAKjhS8jTUXSlMxIJViwb7wK803uayLMPsE8051gu7LP6J+LUwGuZa/T
 0lmBcxQHExvldzIUIFbm
 =nk8f
 -----END PGP SIGNATURE-----

From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: n54@gmx.com, gnats-bugs@NetBSD.org
Cc: pkgsrc-bugs@netbsd.org, tsutsui@ceres.dti.ne.jp
Subject: Re: pkg/50539: pulseaudio consumes too much /var/shm for shm_open(3)
Date: Sun, 13 Dec 2015 05:07:15 +0900

 Kamil Rytarowski wrote:

 >  At first look I think it should be configurable from a config file and
 >  just tuned by user. Perhaps with a MESSAGE warning.

 I notice both client.conf and daemon.conf in /usr/pkg/etc/pulse
 already have the following options:

 ---
 % grep shm /usr/pkg/etc/pulse/*.conf
 /usr/pkg/etc/pulse/client.conf:; enable-shm = yes
 /usr/pkg/etc/pulse/client.conf:; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB
 /usr/pkg/etc/pulse/daemon.conf:; enable-shm = yes
 /usr/pkg/etc/pulse/daemon.conf:; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB
 ---

 so it might be easier to explicitly specify
 >> enable-shm = no
 or
 >> shm-size-bytes = 8388608
 in these config files during make install at least on NetBSD?

 I also see some clients (at least onscripter) don't remove
 the fat shm files in /var/shm on exit.  Umm.

 ---
 Izumi Tsutsui

Responsible-Changed-From-To: pkg-manager->kamil
Responsible-Changed-By: bsiegert@NetBSD.org
Responsible-Changed-When: Mon, 28 Dec 2015 16:17:03 +0000
Responsible-Changed-Why:
Kamil, did your recent changes fix this?


From: Kamil Rytarowski <n54@gmx.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/50539 (pulseaudio consumes too much /var/shm for shm_open(3))
Date: Tue, 29 Dec 2015 00:19:50 +0100

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA256

 On 28.12.2015 17:17, bsiegert@NetBSD.org wrote:
 > Kamil, did your recent changes fix this?

 There is no fix to this as it is.

 I would rather mark it closed as wontfix. It should be discussed with
 upstream developers to reduce the usage of shm_open(3).

 We can still add a note to MESSAGE about this pulseaudio nit.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2

 iQIcBAEBCAAGBQJWgcOUAAoJEEuzCOmwLnZsJYIP/A5X7RAUwp08GA4ChtuHB3gA
 kYLZnz/4dYm3KN9Nzba7gIdq/MWYs0CJOR4jepmFePmdauSYk4KmhLk3j2iTj9zR
 Hqpn7zjmtE+8bBIQmXitQt0nqzIj3s4MZO3glCpPjjOFTqAirs8qq8DZ6e04Z0Vp
 D5bXwvFPXqwX1yOy11tFDE4HJg08QK8mivHw4OAOgjN8KftV5mJeVJLrS+u2zPsX
 Nh+g3p34oEltRYgVnMu/lT55K6vMO1UHCyus7WTuviszTZ1rocsoKTEBz3UGSkxI
 rKMQR0+/1qT9iY0AnPs/3Y0NF4wIMxVHpkvJwMxA15jmU3nurGiJkaLeh5kzfiXT
 utmSThjNiWW2n62TOgheALRJol+OXQPafL2PMN3nLCQbN7wEqXs4Fk7mtkTqnKsG
 pt4gP35tM+O22zpioCpGHFcoh/IzsPXu4PhPGeNngYHU5raZTROgzqV0BJ6RKgD6
 hno06DAp0RB5mYhiU6tTMTat9xMeI/OubYhQpaNdpkyfzPiyU646SPmnCdKXuA9W
 50WTxtzMCHzPdikNIUoFjzaWkBW2Ncwa6jCKIlLT8J0d3YhYu2Ds3SJD8ItIV5wT
 7CRX2fykGCcQdTsiDbBnqHxNZh3hruiCAF4012/J431Nx2o7p2ruw3IDIvbysKuy
 VeZEB2QxajcLHYyyZNEn
 =4WK0
 -----END PGP SIGNATURE-----

State-Changed-From-To: open->suspended
State-Changed-By: bsiegert@NetBSD.org
State-Changed-When: Mon, 28 Dec 2015 23:47:27 +0000
State-Changed-Why:
Needs to be discussed with upstream developers.


From: Arun Raghavan <arun@accosted.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/50539: pulseaudio consumes too much /var/shm for shm_open(3)
Date: Tue, 29 Dec 2015 06:52:14 +0530

 It should be possible for NetBSD to just ship config with the
 appropriate SHM size in daemon.conf. With the upcoming 8.0 release,
 this should be even easier since we support .d directories and you can
 ship a separate override configuration rather than patching the
 installed file.

 Hope this helps.

 -- Arun

Responsible-Changed-From-To: kamil->ryoon
Responsible-Changed-By: kamil@NetBSD.org
Responsible-Changed-When: Tue, 29 Dec 2015 02:39:53 +0100
Responsible-Changed-Why:
Reopen as upstream gave notes in this PR.

Delegate to the maintainer of pulseaudio in pkgsrc as it's pkgsrc specific.
I'm doing NetBSD bits upstream.

Thanks for upstream to have a look at this issue.


State-Changed-From-To: suspended->open
State-Changed-By: kamil@NetBSD.org
State-Changed-When: Tue, 29 Dec 2015 02:39:53 +0100
State-Changed-Why:


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.