NetBSD Problem Report #57816

From www@netbsd.org  Thu Jan  4 15:23:17 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id D64D01A9238
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  4 Jan 2024 15:23:17 +0000 (UTC)
Message-Id: <20240104152316.CA8461A9239@mollari.NetBSD.org>
Date: Thu,  4 Jan 2024 15:23:16 +0000 (UTC)
From: jtocino@gmx.com
Reply-To: jtocino@gmx.com
To: gnats-bugs@NetBSD.org
Subject: Add sysctl support for physical cores
X-Send-Pr-Version: www-1.0

>Number:         57816
>Category:       kern
>Synopsis:       Add sysctl support for physical cores
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Thu Jan 04 15:25:00 +0000 2024
>Last-Modified:  Fri Jan 05 14:10:01 +0000 2024
>Originator:     Jason Bacon
>Release:        9.3
>Organization:
>Environment:
NetBSD netbsd9.acadix.biz 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 5:30:37 UTC 2022  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64

>Description:
NetBSD apparently lacks the ability to report physical cores, as opposed to hyperthreads (hw.ncpu reports hyperthreads).

Darwin reports actual cores as hw.physicalcpu, and FreeBSD as kern.smp.cores.  Both report hyperthreads for hw.ncpy, compatible with NetBSD.

FreeBSD coral.acadix  bacon ~ 1022: sysctl kern.smp
kern.smp.forward_signal_enabled: 1
kern.smp.topology: 0
kern.smp.cores: 2
kern.smp.threads_per_core: 2
kern.smp.cpus: 4
kern.smp.disabled: 0
kern.smp.active: 1
kern.smp.maxcpus: 256
kern.smp.maxid: 3

Darwin tarpon.local  bacon ~ 1003: (pkgsrc): sysctl hw | grep cpu
hw.ncpu: 8
hw.activecpu: 8
hw.perflevel0.physicalcpu: 4
hw.perflevel0.physicalcpu_max: 4
hw.perflevel0.logicalcpu: 4
hw.perflevel0.logicalcpu_max: 4
hw.perflevel0.cpusperl2: 4
hw.perflevel1.physicalcpu: 4
hw.perflevel1.physicalcpu_max: 4
hw.perflevel1.logicalcpu: 4
hw.perflevel1.logicalcpu_max: 4
hw.perflevel1.cpusperl2: 4
hw.physicalcpu: 8
hw.physicalcpu_max: 8
hw.logicalcpu: 8
hw.logicalcpu_max: 8
hw.cputype: 16777228
hw.cpusubtype: 2
hw.cpu64bit_capable: 1
hw.cpufamily: 458787763
hw.cpusubfamily: 2


>How-To-Repeat:
sysctl -a|grep -E 'cpu|core'
>Fix:
Add new sysctls similar to those provided by Darwin or FreeBSD.

>Audit-Trail:
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 16:03:54 -0000 (UTC)

 jtocino@gmx.com writes:

 >Darwin reports actual cores as hw.physicalcpu, and FreeBSD as kern.smp.cores.  Both report hyperthreads for hw.ncpy, compatible with NetBSD.


 That information is currently available using the cpuctl identify command,
 but it's machine specific and probably delivers the information directly
 only for x86.

 For testing I had added sysctls to dump the cpu information in a machine
 independent way, e.g.:

 % sysctl kern.cpu
 kern.cpu.cpu0.package = 0
 kern.cpu.cpu0.core = 0
 kern.cpu.cpu0.smt = 0
 kern.cpu.cpu1.package = 0
 kern.cpu.cpu1.core = 0
 kern.cpu.cpu1.smt = 1
 kern.cpu.cpu2.package = 0
 kern.cpu.cpu2.core = 1
 kern.cpu.cpu2.smt = 0
 kern.cpu.cpu3.package = 0
 kern.cpu.cpu3.core = 1
 kern.cpu.cpu3.smt = 1


 But that's not very useful on machines with many CPUs, it also
 misses newer information about the topology (caches, numa, big.little, ...).

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 17:17:46 +0100

 On Thu, Jan 04, 2024 at 04:05:01PM +0000, Michael van Elst wrote:
 >  That information is currently available using the cpuctl identify command,
 >  but it's machine specific and probably delivers the information directly
 >  only for x86.

 It also has serious downsides:
  - it requires the user to be root
  - it requires one "cpuctl identify N" process for each cpu

 I'm also not sure it is all correct - I have a
 "AMD Ryzen 5 1400 Quad-Core Processor" with cpu0 ... cpu7, and e.g.

 cpu7: AMD Ryzen 5 1400 Quad-Core Processor           , id 0x800f11
 cpu7: node 0, package 0, core 5, smt 1

 (with cores 0, 1, 4 and 5 showing up)

 but "cpuctl identify 7" says:

 cpu7: Initial APIC ID 11
 cpu7: Cluster/Package ID 0
 cpu7: Core ID 1
 cpu7: SMT ID 1

 Martin

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 18:33:38 -0000 (UTC)

 martin@duskware.de (Martin Husemann) writes:

 > On Thu, Jan 04, 2024 at 04:05:01PM +0000, Michael van Elst wrote:
 > >  That information is currently available using the cpuctl identify command,
 > >  but it's machine specific and probably delivers the information directly
 > >  only for x86.
 > 
 > It also has serious downsides:
 >  - it requires the user to be root
 >  - it requires one "cpuctl identify N" process for each cpu

 That's one of the design flaws of cpuctl. All data should be
 collected and provided by the kernel, including the information
 that cannot be collected from MSRs. That should make 'identify'
 a wrapper around sysctl.


 > I'm also not sure it is all correct - I have a
 > "AMD Ryzen 5 1400 Quad-Core Processor" with cpu0 ... cpu7, and e.g.
 > 
 > cpu7: AMD Ryzen 5 1400 Quad-Core Processor           , id 0x800f11
 > cpu7: node 0, package 0, core 5, smt 1
 > 
 > (with cores 0, 1, 4 and 5 showing up)

 The numbers are extracted from the APIC id, and there
 are is some heuristic on how to interpret the different
 bits as package, core or smt (thread) number.

 But these are the numbers the kernel relies on and
 sparse ids are not a problem.


 > but "cpuctl identify 7" says:
 > 
 > cpu7: Initial APIC ID 11
 > cpu7: Cluster/Package ID 0
 > cpu7: Core ID 1
 > cpu7: SMT ID 1

 Even on x86 you cannot read all information from MSRs and
 the code in kernel and cpuctl have diverged for some time.

 Saying that, I don't see the difference here, but the
 calculation of the package id ("socket") is weird:

 Kernel:
         if (smt_bits + core_bits) {
                 if (smt_bits + core_bits < sizeof(apic_id) * NBBY)
                         package_id = apic_id >> (smt_bits + core_bits);
                 else
                         package_id = 0;
         }

 cpuctl:
         ci->ci_packageid = ci->ci_initapicid >> pkg_shift;
         ...
         if (smt_bits + core_bits) {
                 if (smt_bits + core_bits < 32)
                         ci->ci_packageid = 0;
         }

 "diverged"...

From: Jason Bacon <jtocino@gmx.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 13:33:58 -0600

 On 1/4/24 10:05, Michael van Elst wrote:
 > The following reply was made to PR kern/57816; it has been noted by GNAT=
 S.
 >
 > From: mlelstv@serpens.de (Michael van Elst)
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: kern/57816: Add sysctl support for physical cores
 > Date: Thu, 4 Jan 2024 16:03:54 -0000 (UTC)
 >
 >   jtocino@gmx.com writes:
 >
 >   >Darwin reports actual cores as hw.physicalcpu, and FreeBSD as kern.sm=
 p.cores.  Both report hyperthreads for hw.ncpy, compatible with NetBSD.
 >
 >
 >   That information is currently available using the cpuctl identify comm=
 and,
 >   but it's machine specific and probably delivers the information direct=
 ly
 >   only for x86.
 >
 >   For testing I had added sysctls to dump the cpu information in a machi=
 ne
 >   independent way, e.g.:
 >
 >   % sysctl kern.cpu
 >   kern.cpu.cpu0.package =3D 0
 >   kern.cpu.cpu0.core =3D 0
 >   kern.cpu.cpu0.smt =3D 0
 >   kern.cpu.cpu1.package =3D 0
 >   kern.cpu.cpu1.core =3D 0
 >   kern.cpu.cpu1.smt =3D 1
 >   kern.cpu.cpu2.package =3D 0
 >   kern.cpu.cpu2.core =3D 1
 >   kern.cpu.cpu2.smt =3D 0
 >   kern.cpu.cpu3.package =3D 0
 >   kern.cpu.cpu3.core =3D 1
 >   kern.cpu.cpu3.smt =3D 1
 >
 >
 >   But that's not very useful on machines with many CPUs, it also
 >   misses newer information about the topology (caches, numa, big.little,=
  ...).
 >

 I think it would be useful to have another variable for the number of
 physical cores.  It could produce the same value as hw.ncpu on most
 architectures, which don't have hyperthreading.

 Or, if there were a boolean to indicate whether hyperthreading is active
 (which would always be 0 on most architectures), it would be simple
 enough to divide hw.ncpu by 2.

 Or, a threads_per_core variable, which would always be 1 on most archs,
 and 2 where hyperthreading is available and active.

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Thu, 4 Jan 2024 21:36:04 -0000 (UTC)

 jtocino@gmx.com (Jason Bacon) writes:

 >I think it would be useful to have another variable for the number of
 >physical cores.  It could produce the same value as hw.ncpu on most
 >architectures, which don't have hyperthreading.

 >Or, if there were a boolean to indicate whether hyperthreading is active
 >(which would always be 0 on most architectures), it would be simple
 >enough to divide hw.ncpu by 2.

 There are forms of "hyperthreading" where you have more than one
 thread per core and with recent CPU designs, you can even have
 different thread counts on each core.

 A simple 'physical core count' isn't sufficient for anything.

From: matthew green <mrg@eterna23.net>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org, jtocino@gmx.com
Subject: re: kern/57816: Add sysctl support for physical cores
Date: Fri, 05 Jan 2024 16:18:01 +1100

 >  > It also has serious downsides:
 >  >  - it requires the user to be root
 >  >  - it requires one "cpuctl identify N" process for each cpu
 >  
 >  That's one of the design flaws of cpuctl. All data should be
 >  collected and provided by the kernel, including the information
 >  that cannot be collected from MSRs. That should make 'identify'
 >  a wrapper around sysctl.

 i wouldn't call it a design flaw of cpuctl, but of the x86
 backend for it :-)

 the arm and arm64 versions work via sysctl, and i think most
 of the sparc and sparc64 ones too.

 it wouldn't be bad to expose a lot of the basic things (eg,
 numa/package/core/smt IDs, maybe some more like base speed)
 in a MI tree like kern.cpu, most of the current ones are in
 machdep trees of course.


 .mrg.

From: Jason Bacon <jtocino@gmx.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: kern/57816: Add sysctl support for physical cores
Date: Fri, 5 Jan 2024 08:05:49 -0600

 On 1/4/24 15:40, Michael van Elst wrote:
 > The following reply was made to PR kern/57816; it has been noted by GNAT=
 S.
 >
 > From: mlelstv@serpens.de (Michael van Elst)
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: kern/57816: Add sysctl support for physical cores
 > Date: Thu, 4 Jan 2024 21:36:04 -0000 (UTC)
 >
 >   jtocino@gmx.com (Jason Bacon) writes:
 >
 >   >I think it would be useful to have another variable for the number of
 >   >physical cores.  It could produce the same value as hw.ncpu on most
 >   >architectures, which don't have hyperthreading.
 >
 >   >Or, if there were a boolean to indicate whether hyperthreading is act=
 ive
 >   >(which would always be 0 on most architectures), it would be simple
 >   >enough to divide hw.ncpu by 2.
 >
 >   There are forms of "hyperthreading" where you have more than one
 >   thread per core and with recent CPU designs, you can even have
 >   different thread counts on each core.
 >
 >   A simple 'physical core count' isn't sufficient for anything.
 >

 A simple core count is sufficient for preventing oversubscription of
 cores, in order to maximize the speed of individual threads by
 eliminating a major source of contention.  This is the goal of most HPC
 resource managers, and of the py-joblib developers, which triggered this
 discussion (adding BSD support for auto-detection of available cores).

Home	PR Database Search