NetBSD Problem Report #53273
From soda@sra.co.jp Thu May 10 02:41:00 2018
Return-Path: <soda@sra.co.jp>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id B38517A1F7
for <gnats-bugs@gnats.NetBSD.org>; Thu, 10 May 2018 02:41:00 +0000 (UTC)
Message-Id: <20180510021928.40C6B16D4FC1@srapc2586.sra.co.jp>
Date: Thu, 10 May 2018 11:19:28 +0900 (JST)
From: soda@NetBSD.org
Reply-To: soda@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: sem_init() with pshared=1 does not actually work
X-Send-Pr-Version: 3.95
>Number: 53273
>Category: lib
>Synopsis: sem_init() with pshared=1 does not actually work
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: thorpej
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Thu May 10 02:45:00 +0000 2018
>Closed-Date: Sun Feb 03 18:59:37 +0000 2019
>Last-Modified: Tue Feb 05 03:00:01 +0000 2019
>Originator: SODA Noriyuki
>Release: NetBSD 7.1.2
>Organization:
the NetBSD project
>Environment:
System: NetBSD alec 7.1.2 NetBSD 7.1.2 (GENERIC.201803151611Z) amd64
Architecture: x86_64
Machine: amd64
>Description:
if sem_t is created by sem_init(*, 1, *), the sem_t has to be sharable
between processes, even if the processes don't share virtual memory space.
but our implemention doesn't satisfy this requirement.
>How-To-Repeat:
maya@ found this problem by trying to enable the multi-process feature
of recent firefox.
>Fix:
change the definition of sem_t from an opaque pointer to an actual struct
just like other UNIXes.
e.g.
Solaris 9:
typedef struct {
/* this structure must be the same as sema_t in <synch.h> */
uint32_t sem_count; /* semaphore count */
uint16_t sem_type;
uint16_t sem_magic;
upad64_t sem_pad1[3]; /* reserved for a mutex_t */
upad64_t sem_pad2[2]; /* reserved for a cond_t */
} sem_t;
FreeBSD-current:
struct _sem {
__uint32_t _magic;
struct _usem2 _kern;
__uint32_t _padding; /* Preserve structure size */
};
typedef struct _sem sem_t;
Linux:
typedef union
{
char __size[__SIZEOF_SEM_T];
long int __align;
} sem_t;
>Release-Note:
>Audit-Trail:
From: SODA Noriyuki <soda@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Thu, 10 May 2018 12:04:54 +0900
>>>>> On Thu, 10 May 2018 02:45:00 +0000 (UTC), soda@NetBSD.org said:
> change the definition of sem_t from an opaque pointer to an actual struct
> just like other UNIXes.
Or, just using kernel semid as sem_t may be an option,
although I'm not sure whether that causes any problem or not.
But jmcneill@ pointed out that macOS is using such implementation.
i.e.
typedef int sem_t;
From: SODA Noriyuki <soda@NetBSD.org>
To: gnats-bugs@NetBSD.org,
lib-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc:
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Thu, 10 May 2018 12:34:10 +0900
>>>>> On Thu, 10 May 2018 12:04:54 +0900, SODA Noriyuki <soda@NetBSD.org> said:
> Or, just using kernel semid as sem_t may be an option,
> although I'm not sure whether that causes any problem or not.
Currently what ksem_init() returns is just a file descriptor,
and that cannot be shared between processes.
So, not only library, but also kernel has to be changed too.
Thus, the "lib/" category of this PR seems wrong. ;-)
From: coypu@sdf.org
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Thu, 10 May 2018 13:29:08 +0000
for context, the program to use this API and suffer hard from its
brokenness is firefox which crashes often at runtime.
There was some attempt to patch it in pkg/51966 which shouldn't
have helped (it's for multiprocess, so pthread is no good) but
seemed like it did.
commenting in www/firefox/distinfo:
-SHA1 (patch-ipc_glue_CrossProcessSemaphore.h) = 778a569887d8ad082abc2fa2fe89a0a943e84d64
-SHA1 (patch-ipc_glue_CrossProcessSemaphore__posix.cpp) = 3052338871447cedaeafe8369afe82233183c884
+#SHA1 (patch-ipc_glue_CrossProcessSemaphore.h) = 778a569887d8ad082abc2fa2fe89a0a943e84d64
+#SHA1 (patch-ipc_glue_CrossProcessSemaphore__posix.cpp) = 3052338871447cedaeafe8369afe82233183c884
Will make it crash very quickly from lack of locking.
From: Ryo ONODERA <ryo@tetera.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Thu, 06 Sep 2018 00:50:19 +0900 (JST)
Hi,
This problem blocks Chromium web browser porting to NetBSD-current.
Could anyone can handle this problem?
Thank you.
--
Ryo ONODERA // ryo@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB FD1B F404 27FA C7D1 15F3
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Fri, 28 Sep 2018 19:45:40 +0000
On Thu, May 10, 2018 at 02:45:00AM +0000, soda@NetBSD.org wrote:
> if sem_t is created by sem_init(*, 1, *), the sem_t has to be sharable
> between processes, even if the processes don't share virtual memory space.
> but our implemention doesn't satisfy this requirement.
>
> >Fix:
>
> change the definition of sem_t from an opaque pointer to an actual struct
> just like other UNIXes.
I'm not clear on how that will help - is the idea that you're supposed
to be able to write the bits of the struct through a pipe to another
process and have it work on the other end? That seems very unlikely in
general; also if it were necessary the code wanting it almost
certainly wouldn't compile with our implementation.
The man page for sem_init is not clear about this. Can someone
explain? (And while we're at it, the man page for sem_open does not
explain how multiple opens of the same semaphore are supposed to
interact with sem_close, which it should.)
My understanding has always been that if you want to use the same
posix ipc object in multiple processes that you need to name it and
look it up by name in each (with sem_open)...
--
David A. Holland
dholland@netbsd.org
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Sat, 29 Sep 2018 05:01:00 +0700
Date: Fri, 28 Sep 2018 19:50:01 +0000 (UTC)
From: David Holland <dholland-bugs@netbsd.org>
Message-ID: <20180928195001.B7B2E7A1F7@mollari.NetBSD.org>
| I'm not clear on how that will help - is the idea that you're supposed
| to be able to write the bits of the struct through a pipe to another
| process and have it work on the other end?
I think the assumption is that the semaphore sits in shared memory.
Either shared amongst threads (lwps) or between processes (and I
doubt it matters if it is shm() or mmap() generated).
| (And while we're at it, the man page for sem_open does not
| explain how multiple opens of the same semaphore are supposed to
| interact with sem_close, which it should.)
Think of sem_open() and sem_slose() as analogs of open() and close()
with the difference being that they operate on a different namespace,
and the operations available on the returned object differ (no read/write/...
but sem_wait(), sem_post() etc)..
| My understanding has always been that if you want to use the same
| posix ipc object in multiple processes that you need to name it and
| look it up by name in each (with sem_open)...
No, what posix says is ...
any process that can access the semaphore sem can use sem for performing
having the sem in shared mem is what enables other processes to access it.
kre
From: Jason Thorpe <thorpej@me.com>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
soda@NetBSD.org
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Fri, 28 Sep 2018 17:41:26 -0700
Just because it can be accessed doesn=E2=80=99t mean it can work. In NetBSD=
, pshared semaphores probably require kernel support.
-- thorpej
Sent from my iPhone.
> On Sep 28, 2018, at 3:05 PM, Robert Elz <kre@munnari.OZ.AU> wrote:
>=20
> having the sem in shared mem is what enables other processes to access it.=
From: Robert Elz <kre@munnari.OZ.AU>
To: Jason Thorpe <thorpej@me.com>
Cc: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, soda@NetBSD.org
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Sat, 29 Sep 2018 08:19:32 +0700
Date: Fri, 28 Sep 2018 17:41:26 -0700
From: Jason Thorpe <thorpej=40me.com>
Message-ID: <9FB47A7B-61CD-4726-92C1-94F75862BAF0=40me.com>
=7C In NetBSD, pshared semaphores probably require kernel support.
Yes, they certainly will, though it might be possible to do many simple o=
ps
(no contention) without innvoking it.
kre
Responsible-Changed-From-To: lib-bug-people->thorpej
Responsible-Changed-By: thorpej@NetBSD.org
Responsible-Changed-When: Sun, 27 Jan 2019 20:30:11 +0000
Responsible-Changed-Why:
Take.
State-Changed-From-To: open->closed
State-Changed-By: thorpej@NetBSD.org
State-Changed-When: Sun, 03 Feb 2019 18:59:37 +0000
State-Changed-Why:
This is now fixed in 8.99.34.
From: soda@NetBSD.org
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: lib/53273: sem_init() with pshared=1 does not actually work
Date: Tue, 5 Feb 2019 11:56:57 +0900
>>>>> On Thu, 10 May 2018 02:45:00 +0000 (UTC), soda@NetBSD.org said:
> even if the processes don't share virtual memory space.
This sentence was not accurate.
Correct expression is:
even if the processes don't use same virtual memory space.
Acutally, the POSIX specification implicitly assumes that the
processes share same memory (via mmap or something).
Although it doesn't assumes that the processes don't use same
memory space (like parent and fork()ed children).
--
soda
>Unformatted:
-current has same problem, too
(Contact us)
$NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.