NetBSD Problem Report #57816
From www@netbsd.org Thu Jan 4 15:23:17 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id D64D01A9238
for <gnats-bugs@gnats.NetBSD.org>; Thu, 4 Jan 2024 15:23:17 +0000 (UTC)
Message-Id: <20240104152316.CA8461A9239@mollari.NetBSD.org>
Date: Thu, 4 Jan 2024 15:23:16 +0000 (UTC)
From: jtocino@gmx.com
Reply-To: jtocino@gmx.com
To: gnats-bugs@NetBSD.org
Subject: Add sysctl support for physical cores
X-Send-Pr-Version: www-1.0
>Number: 57816
>Category: kern
>Synopsis: Add sysctl support for physical cores
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: kern-bug-people
>State: open
>Class: change-request
>Submitter-Id: net
>Arrival-Date: Thu Jan 04 15:25:00 +0000 2024
>Last-Modified: Fri Jan 05 14:10:01 +0000 2024
>Originator: Jason Bacon
>Release: 9.3
>Organization:
>Environment:
NetBSD netbsd9.acadix.biz 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug 4 5:30:37 UTC 2022 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64
>Description:
NetBSD apparently lacks the ability to report physical cores, as opposed to hyperthreads (hw.ncpu reports hyperthreads).
Darwin reports actual cores as hw.physicalcpu, and FreeBSD as kern.smp.cores. Both report hyperthreads for hw.ncpy, compatible with NetBSD.
FreeBSD coral.acadix bacon ~ 1022: sysctl kern.smp
kern.smp.forward_signal_enabled: 1
kern.smp.topology: 0
kern.smp.cores: 2
kern.smp.threads_per_core: 2
kern.smp.cpus: 4
kern.smp.disabled: 0
kern.smp.active: 1
kern.smp.maxcpus: 256
kern.smp.maxid: 3
Darwin tarpon.local bacon ~ 1003: (pkgsrc): sysctl hw | grep cpu
hw.ncpu: 8
hw.activecpu: 8
hw.perflevel0.physicalcpu: 4
hw.perflevel0.physicalcpu_max: 4
hw.perflevel0.logicalcpu: 4
hw.perflevel0.logicalcpu_max: 4
hw.perflevel0.cpusperl2: 4
hw.perflevel1.physicalcpu: 4
hw.perflevel1.physicalcpu_max: 4
hw.perflevel1.logicalcpu: 4
hw.perflevel1.logicalcpu_max: 4
hw.perflevel1.cpusperl2: 4
hw.physicalcpu: 8
hw.physicalcpu_max: 8
hw.logicalcpu: 8
hw.logicalcpu_max: 8
hw.cputype: 16777228
hw.cpusubtype: 2
hw.cpu64bit_capable: 1
hw.cpufamily: 458787763
hw.cpusubfamily: 2
>How-To-Repeat:
sysctl -a|grep -E 'cpu|core'
>Fix:
Add new sysctls similar to those provided by Darwin or FreeBSD.
>Audit-Trail:
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 16:03:54 -0000 (UTC)
jtocino@gmx.com writes:
>Darwin reports actual cores as hw.physicalcpu, and FreeBSD as kern.smp.cores. Both report hyperthreads for hw.ncpy, compatible with NetBSD.
That information is currently available using the cpuctl identify command,
but it's machine specific and probably delivers the information directly
only for x86.
For testing I had added sysctls to dump the cpu information in a machine
independent way, e.g.:
% sysctl kern.cpu
kern.cpu.cpu0.package = 0
kern.cpu.cpu0.core = 0
kern.cpu.cpu0.smt = 0
kern.cpu.cpu1.package = 0
kern.cpu.cpu1.core = 0
kern.cpu.cpu1.smt = 1
kern.cpu.cpu2.package = 0
kern.cpu.cpu2.core = 1
kern.cpu.cpu2.smt = 0
kern.cpu.cpu3.package = 0
kern.cpu.cpu3.core = 1
kern.cpu.cpu3.smt = 1
But that's not very useful on machines with many CPUs, it also
misses newer information about the topology (caches, numa, big.little, ...).
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 17:17:46 +0100
On Thu, Jan 04, 2024 at 04:05:01PM +0000, Michael van Elst wrote:
> That information is currently available using the cpuctl identify command,
> but it's machine specific and probably delivers the information directly
> only for x86.
It also has serious downsides:
- it requires the user to be root
- it requires one "cpuctl identify N" process for each cpu
I'm also not sure it is all correct - I have a
"AMD Ryzen 5 1400 Quad-Core Processor" with cpu0 ... cpu7, and e.g.
cpu7: AMD Ryzen 5 1400 Quad-Core Processor , id 0x800f11
cpu7: node 0, package 0, core 5, smt 1
(with cores 0, 1, 4 and 5 showing up)
but "cpuctl identify 7" says:
cpu7: Initial APIC ID 11
cpu7: Cluster/Package ID 0
cpu7: Core ID 1
cpu7: SMT ID 1
Martin
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 18:33:38 -0000 (UTC)
martin@duskware.de (Martin Husemann) writes:
> On Thu, Jan 04, 2024 at 04:05:01PM +0000, Michael van Elst wrote:
> > That information is currently available using the cpuctl identify command,
> > but it's machine specific and probably delivers the information directly
> > only for x86.
>
> It also has serious downsides:
> - it requires the user to be root
> - it requires one "cpuctl identify N" process for each cpu
That's one of the design flaws of cpuctl. All data should be
collected and provided by the kernel, including the information
that cannot be collected from MSRs. That should make 'identify'
a wrapper around sysctl.
> I'm also not sure it is all correct - I have a
> "AMD Ryzen 5 1400 Quad-Core Processor" with cpu0 ... cpu7, and e.g.
>
> cpu7: AMD Ryzen 5 1400 Quad-Core Processor , id 0x800f11
> cpu7: node 0, package 0, core 5, smt 1
>
> (with cores 0, 1, 4 and 5 showing up)
The numbers are extracted from the APIC id, and there
are is some heuristic on how to interpret the different
bits as package, core or smt (thread) number.
But these are the numbers the kernel relies on and
sparse ids are not a problem.
> but "cpuctl identify 7" says:
>
> cpu7: Initial APIC ID 11
> cpu7: Cluster/Package ID 0
> cpu7: Core ID 1
> cpu7: SMT ID 1
Even on x86 you cannot read all information from MSRs and
the code in kernel and cpuctl have diverged for some time.
Saying that, I don't see the difference here, but the
calculation of the package id ("socket") is weird:
Kernel:
if (smt_bits + core_bits) {
if (smt_bits + core_bits < sizeof(apic_id) * NBBY)
package_id = apic_id >> (smt_bits + core_bits);
else
package_id = 0;
}
cpuctl:
ci->ci_packageid = ci->ci_initapicid >> pkg_shift;
...
if (smt_bits + core_bits) {
if (smt_bits + core_bits < 32)
ci->ci_packageid = 0;
}
"diverged"...
From: Jason Bacon <jtocino@gmx.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 13:33:58 -0600
On 1/4/24 10:05, Michael van Elst wrote:
> The following reply was made to PR kern/57816; it has been noted by GNAT=
S.
>
> From: mlelstv@serpens.de (Michael van Elst)
> To: gnats-bugs@netbsd.org
> Cc:
> Subject: Re: kern/57816: Add sysctl support for physical cores
> Date: Thu, 4 Jan 2024 16:03:54 -0000 (UTC)
>
> jtocino@gmx.com writes:
>
> >Darwin reports actual cores as hw.physicalcpu, and FreeBSD as kern.sm=
p.cores. Both report hyperthreads for hw.ncpy, compatible with NetBSD.
>
>
> That information is currently available using the cpuctl identify comm=
and,
> but it's machine specific and probably delivers the information direct=
ly
> only for x86.
>
> For testing I had added sysctls to dump the cpu information in a machi=
ne
> independent way, e.g.:
>
> % sysctl kern.cpu
> kern.cpu.cpu0.package =3D 0
> kern.cpu.cpu0.core =3D 0
> kern.cpu.cpu0.smt =3D 0
> kern.cpu.cpu1.package =3D 0
> kern.cpu.cpu1.core =3D 0
> kern.cpu.cpu1.smt =3D 1
> kern.cpu.cpu2.package =3D 0
> kern.cpu.cpu2.core =3D 1
> kern.cpu.cpu2.smt =3D 0
> kern.cpu.cpu3.package =3D 0
> kern.cpu.cpu3.core =3D 1
> kern.cpu.cpu3.smt =3D 1
>
>
> But that's not very useful on machines with many CPUs, it also
> misses newer information about the topology (caches, numa, big.little,=
...).
>
I think it would be useful to have another variable for the number of
physical cores. It could produce the same value as hw.ncpu on most
architectures, which don't have hyperthreading.
Or, if there were a boolean to indicate whether hyperthreading is active
(which would always be 0 on most architectures), it would be simple
enough to divide hw.ncpu by 2.
Or, a threads_per_core variable, which would always be 1 on most archs,
and 2 where hyperthreading is available and active.
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 21:36:04 -0000 (UTC)
jtocino@gmx.com (Jason Bacon) writes:
>I think it would be useful to have another variable for the number of
>physical cores. It could produce the same value as hw.ncpu on most
>architectures, which don't have hyperthreading.
>Or, if there were a boolean to indicate whether hyperthreading is active
>(which would always be 0 on most architectures), it would be simple
>enough to divide hw.ncpu by 2.
There are forms of "hyperthreading" where you have more than one
thread per core and with recent CPU designs, you can even have
different thread counts on each core.
A simple 'physical core count' isn't sufficient for anything.
From: matthew green <mrg@eterna23.net>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, jtocino@gmx.com
Subject: re: kern/57816: Add sysctl support for physical cores
Date: Fri, 05 Jan 2024 16:18:01 +1100
> > It also has serious downsides:
> > - it requires the user to be root
> > - it requires one "cpuctl identify N" process for each cpu
>
> That's one of the design flaws of cpuctl. All data should be
> collected and provided by the kernel, including the information
> that cannot be collected from MSRs. That should make 'identify'
> a wrapper around sysctl.
i wouldn't call it a design flaw of cpuctl, but of the x86
backend for it :-)
the arm and arm64 versions work via sysctl, and i think most
of the sparc and sparc64 ones too.
it wouldn't be bad to expose a lot of the basic things (eg,
numa/package/core/smt IDs, maybe some more like base speed)
in a MI tree like kern.cpu, most of the current ones are in
machdep trees of course.
.mrg.
From: Jason Bacon <jtocino@gmx.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Fri, 5 Jan 2024 08:05:49 -0600
On 1/4/24 15:40, Michael van Elst wrote:
> The following reply was made to PR kern/57816; it has been noted by GNAT=
S.
>
> From: mlelstv@serpens.de (Michael van Elst)
> To: gnats-bugs@netbsd.org
> Cc:
> Subject: Re: kern/57816: Add sysctl support for physical cores
> Date: Thu, 4 Jan 2024 21:36:04 -0000 (UTC)
>
> jtocino@gmx.com (Jason Bacon) writes:
>
> >I think it would be useful to have another variable for the number of
> >physical cores. It could produce the same value as hw.ncpu on most
> >architectures, which don't have hyperthreading.
>
> >Or, if there were a boolean to indicate whether hyperthreading is act=
ive
> >(which would always be 0 on most architectures), it would be simple
> >enough to divide hw.ncpu by 2.
>
> There are forms of "hyperthreading" where you have more than one
> thread per core and with recent CPU designs, you can even have
> different thread counts on each core.
>
> A simple 'physical core count' isn't sufficient for anything.
>
A simple core count is sufficient for preventing oversubscription of
cores, in order to maximize the speed of individual threads by
eliminating a major source of contention. This is the goal of most HPC
resource managers, and of the py-joblib developers, which triggered this
discussion (adding BSD support for auto-detection of available cores).
(Contact us)
$NetBSD: query-full-pr,v 1.47 2022/09/11 19:34:41 kim Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2024
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.