NetBSD Problem Report #55781
From www@netbsd.org Wed Nov 4 00:30:09 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 2A8741A921F
for <gnats-bugs@gnats.NetBSD.org>; Wed, 4 Nov 2020 00:30:09 +0000 (UTC)
Message-Id: <20201104003008.36CAE1A9242@mollari.NetBSD.org>
Date: Wed, 4 Nov 2020 00:30:08 +0000 (UTC)
From: nruslan_devel@yahoo.com
Reply-To: nruslan_devel@yahoo.com
To: gnats-bugs@NetBSD.org
Subject: rump_init() does differentiate when all CPUs are initialized
X-Send-Pr-Version: www-1.0
>Number: 55781
>Category: kern
>Synopsis: rump_init() does differentiate when all CPUs are initialized
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Nov 04 00:35:00 +0000 2020
>Last-Modified: Fri Nov 06 04:25:01 +0000 2020
>Originator: Ruslan Nikolaev
>Release: master
>Organization:
Virginia Tech
>Environment:
>Description:
While working on rumprun-smp (github.com/ssrg-vt/rumprun-smp), we found one particular problem with CPU initialization.
The first (bootstrap) CPU should call rump_init(). However, it is not
clear when for other CPUs it is already safe to schedule threads that are placed to the run queue (e.g., due to blocking, etc., possibly also due to events happening in rump_init() itself).
If we allow the scheduler to use other CPUs prematurely, they may not yet be initialized properly causing various bugs and race conditions.
One approach is to let all other CPUs sleep (or busy wait) until the bootsrap CPU fully initializes all these CPUs in rump_init(). Only after that, allow the scheduler to proceed.
Note that the scheduler cannot simply freeze all other CPUs until after rump_init() completes on the bootstrap CPU. There are several issues, e.g., per-CPU initialization of cprng_init_cpu() which should be scheduled on each CPU and/or similar issues.
>How-To-Repeat:
>Fix:
The way we solved it is just by passing an additional callback function to rump_init(). May not necessarily be the best solution but it is not intrusive and does not require significant changes in rump_init():
diff --git a/sys/rump/include/rump/rump.h b/sys/rump/include/rump/rump.h
index 8da2737f6f77..2ff19adab38a 100644
--- a/sys/rump/include/rump/rump.h
+++ b/sys/rump/include/rump/rump.h
@@ -118,6 +118,7 @@ void rump_unschedule(void);
void rump_printevcnts(void);
int rump_daemonize_begin(void);
+int rump_init_callback(void (*cpuinit_callback) (void));
int rump_init(void);
int rump_init_server(const char *);
int rump_daemonize_done(int);
diff --git a/sys/rump/librump/rumpkern/rump.c b/sys/rump/librump/rumpkern/rump.c
index 91f8462f4da4..6397dc6c8952 100644
--- a/sys/rump/librump/rumpkern/rump.c
+++ b/sys/rump/librump/rumpkern/rump.c
@@ -218,7 +218,7 @@ RUMP_COMPONENT(RUMP_COMPONENT_POSTINIT)
#endif /* RUMP_USE_CTOR */
int
-rump_init(void)
+rump_init_callback(void (*cpuinit_callback) (void))
{
char buf[256];
struct timespec bts;
@@ -231,7 +231,7 @@ rump_init(void)
if (rump_inited)
return 0;
else if (rump_inited == -1)
- panic("rump_init: host process restart required");
+ panic("rump_init_callback: host process restart required");
else
rump_inited = 1;
@@ -393,6 +393,9 @@ rump_init(void)
mp_online = true;
+ if (cpuinit_callback)
+ cpuinit_callback();
+
/* CPUs are up. allow kernel threads to run */
rump_thread_allow(NULL);
@@ -496,6 +499,12 @@ rump_init(void)
return 0;
}
+
+int
+rump_init(void)
+{
+ return rump_init_callback(NULL);
+}
/* historic compat */
__strong_alias(rump__init,rump_init);
>Audit-Trail:
From: Christos Zoulas <christos@zoulas.com>
To: Ruslan Nikolaev <nruslan_devel@yahoo.com>
Cc: gnats-bugs@netbsd.org,
kern-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 10:43:01 -0500
--Apple-Mail=_3FDF9937-27B4-4D07-9FD2-749B157D74D8
Content-Type: multipart/alternative;
boundary="Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F"
--Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
> On Nov 3, 2020, at 8:00 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com> =
wrote:
>=20
> Thanks for the feedback! I'll see if I can fix it. In the meantime, I =
also posted one more SMP-related patch: kern/55781
>=20
> We also have rump (glue) files for new drivers such as ixgbe, nvme, =
etc -- we will post them as well.
How does 55781 work? doesn't "rump_inited" prevent the function to be =
entered from other cpus?
=
https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83cd5594d89=
07dd0be97/sys/rump/librump/rumpkern/rump.c#L238 =
<https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83cd5594d8=
907dd0be97/sys/rump/librump/rumpkern/rump.c#L238>
christos
--Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=us-ascii
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br =
class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div =
class=3D"">On Nov 3, 2020, at 8:00 PM, Ruslan Nikolaev <<a =
href=3D"mailto:nruslan_devel@yahoo.com" =
class=3D"">nruslan_devel@yahoo.com</a>> wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" class=3D"">Thanks for the feedback! I'll =
see if I can fix it. In the meantime, I also posted one more SMP-related =
patch: kern/55781</span><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">We also have rump (glue) files for new drivers such as ixgbe, =
nvme, etc -- we will post them as well.</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""></div></blockquote></div><br =
class=3D""><div class=3D"">How does 55781 work? doesn't "rump_inited" =
prevent the function to be entered from other cpus?</div><div =
class=3D""><br class=3D""></div><div class=3D""><a =
href=3D"https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83c=
d5594d8907dd0be97/sys/rump/librump/rumpkern/rump.c#L238" =
class=3D"">https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f=
83cd5594d8907dd0be97/sys/rump/librump/rumpkern/rump.c#L238</a></div><div =
class=3D""><br class=3D""></div><div =
class=3D"">christos</div></body></html>=
--Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F--
--Apple-Mail=_3FDF9937-27B4-4D07-9FD2-749B157D74D8
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCX6LMBQAKCRBxESqxbLM7
OmiIAKCm43T19qB6si4sSJrGNQci1wUf0QCfVHOsCxNcOHq4InWooDFOlaVO2cQ=
=1VEz
-----END PGP SIGNATURE-----
--Apple-Mail=_3FDF9937-27B4-4D07-9FD2-749B157D74D8--
From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 12:42:21 -0500
No, Christos, the point is different here.
Only the bootstrap CPU calls rump_init() even after our change.
However, during rump_init() other CPUs are already scheduled some tasks,
so we need to wake them up, we cannot simply wait until rump_init()
finishes. If we wake them too early (i.e., before rump_init()), their
state will not be properly initialized yet. So, we need to wake them up
while in rump_init() but only *after* their state is fully initialized.
So, this is when the bootstrap CPU will call the callback function.
Note that other CPUs will not call rump_init(), they will simply
schedule some queued tasks.
Does it make sense?
Ruslan
On 11/4/20 10:43 AM, Christos Zoulas wrote:
>
>
>> On Nov 3, 2020, at 8:00 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com
>> <mailto:nruslan_devel@yahoo.com>> wrote:
>>
>> Thanks for the feedback! I'll see if I can fix it. In the meantime, I
>> also posted one more SMP-related patch: kern/55781
>>
>> We also have rump (glue) files for new drivers such as ixgbe, nvme,
>> etc -- we will post them as well.
>
> How does 55781 work? doesn't "rump_inited" prevent the function to be
> entered from other cpus?
>
> https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83cd5594d8907dd0be97/sys/rump/librump/rumpkern/rump.c#L238
>
> christos
From: Christos Zoulas <christos@zoulas.com>
To: Ruslan Nikolaev <nruslan_devel@yahoo.com>
Cc: gnats-bugs@netbsd.org,
kern-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 12:49:21 -0500
--Apple-Mail=_5E240126-329A-4497-9786-4E4C75FE017D
Content-Type: multipart/alternative;
boundary="Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529"
--Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
> On Nov 4, 2020, at 12:42 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com> =
wrote:
>=20
> No, Christos, the point is different here.
>=20
> Only the bootstrap CPU calls rump_init() even after our change.
>=20
> However, during rump_init() other CPUs are already scheduled some =
tasks, so we need to wake them up, we cannot simply wait until =
rump_init() finishes. If we wake them too early (i.e., before =
rump_init()), their state will not be properly initialized yet. So, we =
need to wake them up while in rump_init() but only *after* their state =
is fully initialized.
>=20
> So, this is when the bootstrap CPU will call the callback function.
>=20
>=20
> Note that other CPUs will not call rump_init(), they will simply =
schedule some queued tasks.
>=20
>=20
> Does it make sense?
Yes, but where is rump_init_callback() called with a non-NULL argument?
christos
--Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=us-ascii
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br =
class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div =
class=3D"">On Nov 4, 2020, at 12:42 PM, Ruslan Nikolaev <<a =
href=3D"mailto:nruslan_devel@yahoo.com" =
class=3D"">nruslan_devel@yahoo.com</a>> wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" class=3D"">No, Christos, the point is =
different here.</span><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">Only the bootstrap CPU calls rump_init() even after our =
change.</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none;" class=3D""><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
none; float: none; display: inline !important;" class=3D"">However, =
during rump_init() other CPUs are already scheduled some tasks, so we =
need to wake them up, we cannot simply wait until rump_init() finishes. =
If we wake them too early (i.e., before rump_init()), their state will =
not be properly initialized yet. So, we need to wake them up while in =
rump_init() but only *after* their state is fully initialized.</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: =
normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;" class=3D"">So, this is when the bootstrap =
CPU will call the callback function.</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">Note that other CPUs will not call rump_init(), they will =
simply schedule some queued tasks.</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;" =
class=3D"">Does it make sense?</span><br style=3D"caret-color: rgb(0, 0, =
0); font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;" class=3D""></div></blockquote></div><br =
class=3D""><div class=3D"">Yes, but where is rump_init_callback() called =
with a non-NULL argument?</div><div class=3D""><br class=3D""></div><div =
class=3D"">christos</div></body></html>=
--Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529--
--Apple-Mail=_5E240126-329A-4497-9786-4E4C75FE017D
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCX6LpogAKCRBxESqxbLM7
Os2fAKCgthLY0UahAR0J7sQYWm4CnACUDwCghTMlGa0Ql8lXhmXBH8W72kXUWw4=
=tiET
-----END PGP SIGNATURE-----
--Apple-Mail=_5E240126-329A-4497-9786-4E4C75FE017D--
From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 13:47:21 -0500
This is called *in lieu* of rump_init() for > 1 CPU (for SMP when a
callback function to wake up sleeping CPUs is needed).
Basically, it should be new rump_init() but there is plenty of code
which calls it in the current source tree
On 11/4/20 12:49 PM, Christos Zoulas wrote:
>
>
>> On Nov 4, 2020, at 12:42 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com
>> <mailto:nruslan_devel@yahoo.com>> wrote:
>>
>> No, Christos, the point is different here.
>>
>> Only the bootstrap CPU calls rump_init() even after our change.
>>
>> However, during rump_init() other CPUs are already scheduled some
>> tasks, so we need to wake them up, we cannot simply wait until
>> rump_init() finishes. If we wake them too early (i.e., before
>> rump_init()), their state will not be properly initialized yet. So,
>> we need to wake them up while in rump_init() but only *after* their
>> state is fully initialized.
>>
>> So, this is when the bootstrap CPU will call the callback function.
>>
>>
>> Note that other CPUs will not call rump_init(), they will simply
>> schedule some queued tasks.
>>
>>
>> Does it make sense?
>
> Yes, but where is rump_init_callback() called with a non-NULL argument?
>
> christos
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/55781 CVS commit: src/sys/rump
Date: Wed, 4 Nov 2020 17:06:39 -0500
Module Name: src
Committed By: christos
Date: Wed Nov 4 22:06:39 UTC 2020
Modified Files:
src/sys/rump/include/rump: rump.h
src/sys/rump/librump/rumpkern: rump.c
Log Message:
PR/55781: Ruslan Nikolaev: rump_init() does differentiate when all CPUs are
initialized
To generate a diff of this commit:
cvs rdiff -u -r1.72 -r1.73 src/sys/rump/include/rump/rump.h
cvs rdiff -u -r1.349 -r1.350 src/sys/rump/librump/rumpkern/rump.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Thu, 5 Nov 2020 23:21:46 -0500
Thanks, Christos for merging this change. I also updated kern/55777 with
a new patch. I have also created kern/55789 for rump drivers patches.
On 11/4/20 12:49 PM, Christos Zoulas wrote:
>
> Yes, but where is rump_init_callback() called with a non-NULL argument?
>
> christos
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.