NetBSD Problem Report #54431
From he@smistad.uninett.no Fri Aug 2 11:31:23 2019
Return-Path: <he@smistad.uninett.no>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 23BEA7A189
for <gnats-bugs@gnats.NetBSD.org>; Fri, 2 Aug 2019 11:31:23 +0000 (UTC)
Message-Id: <20190802113117.9AB1F43F4E9@smistad.uninett.no>
Date: Fri, 2 Aug 2019 13:31:17 +0200 (CEST)
From: he@NetBSD.org
Reply-To: he@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: Missing 64-bit atomics on 32-bit platforms
X-Send-Pr-Version: 3.95
>Number: 54431
>Category: toolchain
>Synopsis: Missing 64-bit atomics on 32-bit platforms
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: toolchain-manager
>State: open
>Class: support
>Submitter-Id: net
>Arrival-Date: Fri Aug 02 11:35:00 +0000 2019
>Last-Modified: Sat Aug 03 01:50:01 +0000 2019
>Originator: he@NetBSD.org
>Release: NetBSD 8.0
>Organization:
I try...
>Environment:
System: NetBSD ambrosia.urc.uninett.no 8.0 NetBSD 8.0 (AMBROSIA) #4: Tue Apr 10 09:30:29 CEST 2018 he@ambrosia.urc.uninett.no:/usr/obj/sys/arch/macppc/compile/AMBROSIA macppc
Architecture: powerpc
Machine: macppc
>Description:
This is vaguely related to PR#54428.
It appears that at least our powerpc and i386 ports do not
support any sort of 64-bit atomic operations.
This has already caused us to carry around patches to llvm and
rust to not insist on the presence of 64-bit atomic
operations.
The latest thing I stumbled across was in an attempt to update
wip/quickjs to the new 2019-07-28 snapshot, and building that
on NetBSD/macppc 8.0 fails with
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48182: undefined reference to `__atomic_compare_exchange_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48138: undefined reference to `__atomic_fetch_xor_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48139: undefined reference to `__atomic_exchange_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48153: undefined reference to `__atomic_load_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48134: undefined reference to `__atomic_fetch_add_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48135: undefined reference to `__atomic_fetch_and_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48136: undefined reference to `__atomic_fetch_or_8'
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48137: undefined reference to `__atomic_fetch_sub_8'
.obj/quickjs.bn.o: In function `js_atomics_store':
/usr/pkgsrc/wip/quickjs/work/quickjs-2019-07-28/quickjs.c:48246: undefined reference to `__atomic_store_8'
gmake: *** [Makefile:180: qjsbnc] Error 1
(Not yet mentioned to the author, I'll try to do that...)
However, I'm quite certain these are not the only ones which
more or less assume the presence of 64-bit atomics.
Other systems appears to solve this problem somehow (provide
primitives for 64-bit atomics even though the native CPU
doesn't provide the instruction-level primitives to do so
directly), but NetBSD does not.
The question is whether it's wise for NetBSD to stick to this
position, and depending on the outcome of that question, what
we need to do about this.
>How-To-Repeat:
Try to update wip/quickjs to the 2019-07-28 version, and build
on either NetBSD/i386 8.0 or NetBSD/macppc 8.0, and watch it
fail as above.
>Fix:
Don't know, sorry.
>Audit-Trail:
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
Date: Fri, 2 Aug 2019 14:27:41 +0200
On Fri, Aug 02, 2019 at 11:35:00AM +0000, he@NetBSD.org wrote:
> It appears that at least our powerpc and i386 ports do not
> support any sort of 64-bit atomic operations.
I do not consider this a bug. The reality is that there is no good way
to provide fall back implementations for larger-than-pointer atomics on
most UP platforms or atomic emulation in general on MP platforms.
We have some hacks on SPARC using hashed locks, but they have major
subtile error cases. Most noticable, they don't work on shared memory.
There is a secondary issue for larger-than-pointer atomics because most
architectures don't have atomic read or write instructions for that
size. At least for the old __sync_* interface that is a major deal
breaker as the load/store is typically written as fence + plain C
volatile load/store.
Any correct implementation will involve a system and hashing of the
physical address of the access memory for larger-than-pointer access for
any operation. I'm pretty sure that is going to kill the performance of
a lot of those users. It doesn't help that there is no way for the user
to promise that a certain memory address is not shared...
Joerg
From: Kamil Rytarowski <n54@gmx.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
Date: Fri, 2 Aug 2019 15:39:13 +0200
On 02.08.2019 14:30, Joerg Sonnenberger wrote:
> The following reply was made to PR toolchain/54431; it has been noted by=
GNATS.
>
> From: Joerg Sonnenberger <joerg@bec.de>
> To: gnats-bugs@netbsd.org
> Cc:
> Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
> Date: Fri, 2 Aug 2019 14:27:41 +0200
>
> On Fri, Aug 02, 2019 at 11:35:00AM +0000, he@NetBSD.org wrote:
> > It appears that at least our powerpc and i386 ports do not
> > support any sort of 64-bit atomic operations.
>
> I do not consider this a bug. The reality is that there is no good way
> to provide fall back implementations for larger-than-pointer atomics on
> most UP platforms or atomic emulation in general on MP platforms.
> We have some hacks on SPARC using hashed locks, but they have major
> subtile error cases. Most noticable, they don't work on shared memory.
> There is a secondary issue for larger-than-pointer atomics because most
> architectures don't have atomic read or write instructions for that
> size. At least for the old __sync_* interface that is a major deal
> breaker as the load/store is typically written as fence + plain C
> volatile load/store.
>
> Any correct implementation will involve a system and hashing of the
> physical address of the access memory for larger-than-pointer access fo=
r
> any operation. I'm pretty sure that is going to kill the performance of
> a lot of those users. It doesn't help that there is no way for the user
> to promise that a certain memory address is not shared...
>
> Joerg
>
>
We already ship with ras(9) that is significantly worse and does not
work in probably any SMP or shared memory scenarios.. but we ship with
it basically in every port as in UP setups it can be good enough.
libatomic in general works unless we are in fewer corner-cases.
SPARC implementation could be improved if needed and should not block
ppc or i386.
More modern i386 (>=3D 586) supports 64bit atomics natively with cmpxchg8b=
.
From: Jason Thorpe <thorpej@me.com>
To: gnats-bugs@netbsd.org
Cc: toolchain-manager@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
"he@netbsd.org" <he@NetBSD.org>
Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
Date: Fri, 2 Aug 2019 07:16:55 -0700
> On Aug 2, 2019, at 5:30 AM, Joerg Sonnenberger <joerg@bec.de> wrote:
>=20
> Any correct implementation will involve a system and hashing of the
> physical address of the access memory for larger-than-pointer access =
for
> any operation. I'm pretty sure that is going to kill the performance =
of
> a lot of those users. It doesn't help that there is no way for the =
user
> to promise that a certain memory address is not shared...
There are other implementations possible, but yah, they could be really =
slow. But maybe slow-but-hey-it-works-at-least is OK?
-- thorpej
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc: toolchain-manager@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, he@NetBSD.org
Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
Date: Sat, 3 Aug 2019 02:44:30 +0200
On Fri, Aug 02, 2019 at 01:40:01PM +0000, Kamil Rytarowski wrote:
> We already ship with ras(9) that is significantly worse and does not
> work in probably any SMP or shared memory scenarios.. but we ship with
> it basically in every port as in UP setups it can be good enough.
ras(9) is nearly zero overhead and *only* meant to help implementing
atomics on UP architectures. It doesn't care about shared memory at all.
> libatomic in general works unless we are in fewer corner-cases.
I do not call shared memory a corner case. Given that one of the reasons
that certain folks are loudly pushing for futex semantics everywhere is
because they want to implement locking primitives in shared memory, that
shouldn't be surprising at all. Given that double width cas is the
typical solution for dealing with lock-free data structures, even less.
> SPARC implementation could be improved if needed and should not block
> ppc or i386.
I meanted Sparc because it is the one architecture with a non-trivial
deployment of SMP systems and without native CAS or LL/SC. I don't count
VAX. That said, the problems of Sparc atomics is shared with 64bit
atomics on 32bit PPC and i386 as well as 128bit atomics on most LP64
platforms.
Joerg
From: Kamil Rytarowski <n54@gmx.com>
To: "gnats-bugs@NetBSD.org" <gnats-bugs@NetBSD.org>
Cc:
Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
Date: Sat, 3 Aug 2019 03:46:10 +0200
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--I4ptJEDr2abzWe6gQns0M4eqtNszrE1Y5
Content-Type: multipart/mixed; boundary="nCr0hHIhWCUhGTUoQz54UCM092GVXeHbN";
protected-headers="v1"
From: Kamil Rytarowski <n54@gmx.com>
To: "gnats-bugs@NetBSD.org" <gnats-bugs@NetBSD.org>
Message-ID: <6d527e62-6019-2171-e043-92b0c0a3aa45@gmx.com>
Subject: Re: toolchain/54431: Missing 64-bit atomics on 32-bit platforms
References: <pr-toolchain-54431@gnats.netbsd.org>
<20190802113117.9AB1F43F4E9@smistad.uninett.no>
<20190802134001.AE80D7A1C0@mollari.NetBSD.org>
<20190803004430.GB23485@bec.de>
In-Reply-To: <20190803004430.GB23485@bec.de>
--nCr0hHIhWCUhGTUoQz54UCM092GVXeHbN
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
On 03.08.2019 02:44, Joerg Sonnenberger wrote:
> On Fri, Aug 02, 2019 at 01:40:01PM +0000, Kamil Rytarowski wrote:
>> We already ship with ras(9) that is significantly worse and does not
>> work in probably any SMP or shared memory scenarios.. but we ship wit=
h
>> it basically in every port as in UP setups it can be good enough.
>=20
> ras(9) is nearly zero overhead and *only* meant to help implementing
> atomics on UP architectures. It doesn't care about shared memory at all=
=2E
>=20
ras(9) having nearly 0 overhead is debatable as it is used in every
cpu_switchto(9) call, even if there are nearly no users of it on modern
CPUs.
Similar with libatomic, users of it do not care about shared memory
unless they are doing something wrongly. No need to push the bar too
high that cannot be solved (without switching to UP kernel mode)
I see no problem with performance hit. It can be really slow, but
running existing unmodified code is a good tradeoff.
>> libatomic in general works unless we are in fewer corner-cases.
>=20
> I do not call shared memory a corner case. Given that one of the reason=
s
> that certain folks are loudly pushing for futex semantics everywhere is=
> because they want to implement locking primitives in shared memory, tha=
t
> shouldn't be surprising at all. Given that double width cas is the
> typical solution for dealing with lock-free data structures, even less.=
>=20
With the switch from libatomic_ops to C11/C++14 atomics there will be a
growing demand for a fallback library implementation (unless we will
stop caring about hardware predating 64/128bit atomic operations).
One of the candidates for libatomic in base is LLDB (I am aware that we
can patch LLDB to stop demanding it).
>> SPARC implementation could be improved if needed and should not block=
>> ppc or i386.
>=20
> I meanted Sparc because it is the one architecture with a non-trivial
> deployment of SMP systems and without native CAS or LL/SC. I don't coun=
t
> VAX. That said, the problems of Sparc atomics is shared with 64bit
> atomics on 32bit PPC and i386 as well as 128bit atomics on most LP64
> platforms.
>=20
With the push toward at least CHERI 128bit pointer architecture, we are
going to have a growing need for 128bit atomics.
They are already in toolchains and allocators.
> Joerg
>=20
--nCr0hHIhWCUhGTUoQz54UCM092GVXeHbN--
--I4ptJEDr2abzWe6gQns0M4eqtNszrE1Y5
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEELaxVpweEzw+lMDwuS7MI6bAudmwFAl1E52IACgkQS7MI6bAu
dmxYXBAAlH7A3YCOvgG8zaJEW2o4UXzlG2w6wlZUhfonMbWTt8tukOGZ1naX3WVE
UbUu/kLrbzwQ3YQKZe4HrZAfgq3fqrxvdOrRAABSEpc7PSAQeFyvxD8oDLbEiT0P
qwGu6zRBvRY6TGMgk6D/2LpbGoSqZGETSaf4sXUt5frAs3CtbyLVk/C7WHIVkP9u
YqV2pSBibemQL8QGIexymvGCN0ZDHfI3zAiXeVaMd8+RlCg9w0YOE8vvCE59tVWU
Grl7Bx06iIxrtvOHCmk8RcsAlADGDA5+o7wSQ8RouIBFUrsXW7/RTDNNIK3Z8yuQ
4Ri7rXeOG+5jSl5mtr6Y5RBbBYxtI6cDc0rQnOd3/NPPFrdaGegCIE0Jf0Pc0IFn
2cw1g+OS7Z4Q47b3hg36NrMueHt0q+Yj5t3pzWe083hMjSK6OYIgF+j4FESKVjUS
tXlfjnHCIsI0WmDiB4h8Ucqd3gABGD/Sp+qpIo8uxRDGQakgwsv+c0IadYwhnsiA
EP80lzZcm4iIB8StNDT3sRs3mXk2PtUhhfQbHIEN26ZXHbEfyw9H2AH07tuLlqsG
kI233bJwntB9ZJCK0D2zgKEjPZfUd6ZA5UJ9eN7fpB4E/8PZctAyspINTPkClSHQ
uf88bQMiLRTvsIs9ZESEMbW17jBMOTdLcUco8OLOs0reHWhjM6A=
=h+EU
-----END PGP SIGNATURE-----
--I4ptJEDr2abzWe6gQns0M4eqtNszrE1Y5--
(Contact us)
$NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.