NetBSD Problem Report #55781

From www@netbsd.org  Wed Nov  4 00:30:09 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 2A8741A921F
	for <gnats-bugs@gnats.NetBSD.org>; Wed,  4 Nov 2020 00:30:09 +0000 (UTC)
Message-Id: <20201104003008.36CAE1A9242@mollari.NetBSD.org>
Date: Wed,  4 Nov 2020 00:30:08 +0000 (UTC)
From: nruslan_devel@yahoo.com
Reply-To: nruslan_devel@yahoo.com
To: gnats-bugs@NetBSD.org
Subject: rump_init() does differentiate when all CPUs are initialized
X-Send-Pr-Version: www-1.0

>Number:         55781
>Category:       kern
>Synopsis:       rump_init() does differentiate when all CPUs are initialized
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Nov 04 00:35:00 +0000 2020
>Last-Modified:  Fri Nov 06 04:25:01 +0000 2020
>Originator:     Ruslan Nikolaev
>Release:        master
>Organization:
Virginia Tech
>Environment:
>Description:
While working on rumprun-smp (github.com/ssrg-vt/rumprun-smp), we found one particular problem with CPU initialization.

The first (bootstrap) CPU should call rump_init(). However, it is not
clear when for other CPUs it is already safe to schedule threads that are placed to the run queue (e.g., due to blocking, etc., possibly also due to events happening in rump_init() itself).
If we allow the scheduler to use other CPUs prematurely, they may not yet be initialized properly causing various bugs and race conditions.

One approach is to let all other CPUs sleep (or busy wait) until the bootsrap CPU fully initializes all these CPUs in rump_init(). Only after that, allow the scheduler to proceed.

Note that the scheduler cannot simply freeze all other CPUs until after rump_init() completes on the bootstrap CPU. There are several issues, e.g., per-CPU initialization of cprng_init_cpu() which should be scheduled on each CPU and/or similar issues.
>How-To-Repeat:

>Fix:
The way we solved it is just by passing an additional callback function to rump_init(). May not necessarily be the best solution but it is not intrusive and does not require significant changes in rump_init():

diff --git a/sys/rump/include/rump/rump.h b/sys/rump/include/rump/rump.h
index 8da2737f6f77..2ff19adab38a 100644
--- a/sys/rump/include/rump/rump.h
+++ b/sys/rump/include/rump/rump.h
@@ -118,6 +118,7 @@ void	rump_unschedule(void);
 void	rump_printevcnts(void);

 int	rump_daemonize_begin(void);
+int	rump_init_callback(void (*cpuinit_callback) (void));
 int	rump_init(void);
 int	rump_init_server(const char *);
 int	rump_daemonize_done(int);
diff --git a/sys/rump/librump/rumpkern/rump.c b/sys/rump/librump/rumpkern/rump.c
index 91f8462f4da4..6397dc6c8952 100644
--- a/sys/rump/librump/rumpkern/rump.c
+++ b/sys/rump/librump/rumpkern/rump.c
@@ -218,7 +218,7 @@ RUMP_COMPONENT(RUMP_COMPONENT_POSTINIT)
 #endif /* RUMP_USE_CTOR */

 int
-rump_init(void)
+rump_init_callback(void (*cpuinit_callback) (void))
 {
 	char buf[256];
 	struct timespec bts;
@@ -231,7 +231,7 @@ rump_init(void)
 	if (rump_inited)
 		return 0;
 	else if (rump_inited == -1)
-		panic("rump_init: host process restart required");
+		panic("rump_init_callback: host process restart required");
 	else
 		rump_inited = 1;

@@ -393,6 +393,9 @@ rump_init(void)

 	mp_online = true;

+	if (cpuinit_callback)
+		cpuinit_callback();
+
 	/* CPUs are up.  allow kernel threads to run */
 	rump_thread_allow(NULL);

@@ -496,6 +499,12 @@ rump_init(void)

 	return 0;
 }
+
+int
+rump_init(void)
+{
+	return rump_init_callback(NULL);
+}
 /* historic compat */
 __strong_alias(rump__init,rump_init);


>Audit-Trail:
From: Christos Zoulas <christos@zoulas.com>
To: Ruslan Nikolaev <nruslan_devel@yahoo.com>
Cc: gnats-bugs@netbsd.org,
 kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 10:43:01 -0500

 --Apple-Mail=_3FDF9937-27B4-4D07-9FD2-749B157D74D8
 Content-Type: multipart/alternative;
 	boundary="Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F"


 --Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/plain;
 	charset=us-ascii



 > On Nov 3, 2020, at 8:00 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com> =
 wrote:
 >=20
 > Thanks for the feedback! I'll see if I can fix it. In the meantime, I =
 also posted one more SMP-related patch: kern/55781
 >=20
 > We also have rump (glue) files for new drivers such as ixgbe, nvme, =
 etc -- we will post them as well.

 How does 55781 work? doesn't "rump_inited" prevent the function to be =
 entered from other cpus?

 =
 https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83cd5594d89=
 07dd0be97/sys/rump/librump/rumpkern/rump.c#L238 =
 <https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83cd5594d8=
 907dd0be97/sys/rump/librump/rumpkern/rump.c#L238>

 christos

 --Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/html;
 	charset=us-ascii

 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
 charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
 -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br =
 class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div =
 class=3D"">On Nov 3, 2020, at 8:00 PM, Ruslan Nikolaev &lt;<a =
 href=3D"mailto:nruslan_devel@yahoo.com" =
 class=3D"">nruslan_devel@yahoo.com</a>&gt; wrote:</div><br =
 class=3D"Apple-interchange-newline"><div class=3D""><span =
 style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
 12px; font-style: normal; font-variant-caps: normal; font-weight: =
 normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
 text-transform: none; white-space: normal; word-spacing: 0px; =
 -webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
 display: inline !important;" class=3D"">Thanks for the feedback! I'll =
 see if I can fix it. In the meantime, I also posted one more SMP-related =
 patch: kern/55781</span><br style=3D"caret-color: rgb(0, 0, 0); =
 font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none; float: none; display: inline !important;" =
 class=3D"">We also have rump (glue) files for new drivers such as ixgbe, =
 nvme, etc -- we will post them as well.</span><br style=3D"caret-color: =
 rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
 normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
 normal; text-align: start; text-indent: 0px; text-transform: none; =
 white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""></div></blockquote></div><br =
 class=3D""><div class=3D"">How does 55781 work? doesn't "rump_inited" =
 prevent the function to be entered from other cpus?</div><div =
 class=3D""><br class=3D""></div><div class=3D""><a =
 href=3D"https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83c=
 d5594d8907dd0be97/sys/rump/librump/rumpkern/rump.c#L238" =
 class=3D"">https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f=
 83cd5594d8907dd0be97/sys/rump/librump/rumpkern/rump.c#L238</a></div><div =
 class=3D""><br class=3D""></div><div =
 class=3D"">christos</div></body></html>=

 --Apple-Mail=_AAFF1261-194E-4F40-A767-FA40CD80CF3F--

 --Apple-Mail=_3FDF9937-27B4-4D07-9FD2-749B157D74D8
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCX6LMBQAKCRBxESqxbLM7
 OmiIAKCm43T19qB6si4sSJrGNQci1wUf0QCfVHOsCxNcOHq4InWooDFOlaVO2cQ=
 =1VEz
 -----END PGP SIGNATURE-----

 --Apple-Mail=_3FDF9937-27B4-4D07-9FD2-749B157D74D8--

From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 12:42:21 -0500

 No, Christos, the point is different here.

 Only the bootstrap CPU calls rump_init() even after our change.

 However, during rump_init() other CPUs are already scheduled some tasks, 
 so we need to wake them up, we cannot simply wait until rump_init() 
 finishes. If we wake them too early (i.e., before rump_init()), their 
 state will not be properly initialized yet. So, we need to wake them up 
 while in rump_init() but only *after* their state is fully initialized.

 So, this is when the bootstrap CPU will call the callback function.


 Note that other CPUs will not call rump_init(), they will simply 
 schedule some queued tasks.


 Does it make sense?


 Ruslan


 On 11/4/20 10:43 AM, Christos Zoulas wrote:
 >
 >
 >> On Nov 3, 2020, at 8:00 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com 
 >> <mailto:nruslan_devel@yahoo.com>> wrote:
 >>
 >> Thanks for the feedback! I'll see if I can fix it. In the meantime, I 
 >> also posted one more SMP-related patch: kern/55781
 >>
 >> We also have rump (glue) files for new drivers such as ixgbe, nvme, 
 >> etc -- we will post them as well.
 >
 > How does 55781 work? doesn't "rump_inited" prevent the function to be 
 > entered from other cpus?
 >
 > https://github.com/ssrg-vt/src-netbsd/blob/168bf207298443ea437f83cd5594d8907dd0be97/sys/rump/librump/rumpkern/rump.c#L238
 >
 > christos

From: Christos Zoulas <christos@zoulas.com>
To: Ruslan Nikolaev <nruslan_devel@yahoo.com>
Cc: gnats-bugs@netbsd.org,
 kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 12:49:21 -0500

 --Apple-Mail=_5E240126-329A-4497-9786-4E4C75FE017D
 Content-Type: multipart/alternative;
 	boundary="Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529"


 --Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/plain;
 	charset=us-ascii



 > On Nov 4, 2020, at 12:42 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com> =
 wrote:
 >=20
 > No, Christos, the point is different here.
 >=20
 > Only the bootstrap CPU calls rump_init() even after our change.
 >=20
 > However, during rump_init() other CPUs are already scheduled some =
 tasks, so we need to wake them up, we cannot simply wait until =
 rump_init() finishes. If we wake them too early (i.e., before =
 rump_init()), their state will not be properly initialized yet. So, we =
 need to wake them up while in rump_init() but only *after* their state =
 is fully initialized.
 >=20
 > So, this is when the bootstrap CPU will call the callback function.
 >=20
 >=20
 > Note that other CPUs will not call rump_init(), they will simply =
 schedule some queued tasks.
 >=20
 >=20
 > Does it make sense?

 Yes, but where is rump_init_callback() called with a non-NULL argument?

 christos

 --Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/html;
 	charset=us-ascii

 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
 charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
 -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br =
 class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div =
 class=3D"">On Nov 4, 2020, at 12:42 PM, Ruslan Nikolaev &lt;<a =
 href=3D"mailto:nruslan_devel@yahoo.com" =
 class=3D"">nruslan_devel@yahoo.com</a>&gt; wrote:</div><br =
 class=3D"Apple-interchange-newline"><div class=3D""><span =
 style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
 12px; font-style: normal; font-variant-caps: normal; font-weight: =
 normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
 text-transform: none; white-space: normal; word-spacing: 0px; =
 -webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
 display: inline !important;" class=3D"">No, Christos, the point is =
 different here.</span><br style=3D"caret-color: rgb(0, 0, 0); =
 font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none; float: none; display: inline !important;" =
 class=3D"">Only the bootstrap CPU calls rump_init() even after our =
 change.</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
 Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
 normal; font-weight: normal; letter-spacing: normal; text-align: start; =
 text-indent: 0px; text-transform: none; white-space: normal; =
 word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
 none;" class=3D""><br style=3D"caret-color: rgb(0, 0, 0); font-family: =
 Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
 normal; font-weight: normal; letter-spacing: normal; text-align: start; =
 text-indent: 0px; text-transform: none; white-space: normal; =
 word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
 none;" class=3D""><span style=3D"caret-color: rgb(0, 0, 0); font-family: =
 Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
 normal; font-weight: normal; letter-spacing: normal; text-align: start; =
 text-indent: 0px; text-transform: none; white-space: normal; =
 word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
 none; float: none; display: inline !important;" class=3D"">However, =
 during rump_init() other CPUs are already scheduled some tasks, so we =
 need to wake them up, we cannot simply wait until rump_init() finishes. =
 If we wake them too early (i.e., before rump_init()), their state will =
 not be properly initialized yet. So, we need to wake them up while in =
 rump_init() but only *after* their state is fully initialized.</span><br =
 style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
 12px; font-style: normal; font-variant-caps: normal; font-weight: =
 normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
 text-transform: none; white-space: normal; word-spacing: 0px; =
 -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><br =
 style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
 12px; font-style: normal; font-variant-caps: normal; font-weight: =
 normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
 text-transform: none; white-space: normal; word-spacing: 0px; =
 -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span =
 style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
 12px; font-style: normal; font-variant-caps: normal; font-weight: =
 normal; letter-spacing: normal; text-align: start; text-indent: 0px; =
 text-transform: none; white-space: normal; word-spacing: 0px; =
 -webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
 display: inline !important;" class=3D"">So, this is when the bootstrap =
 CPU will call the callback function.</span><br style=3D"caret-color: =
 rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
 normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
 normal; text-align: start; text-indent: 0px; text-transform: none; =
 white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none; float: none; display: inline !important;" =
 class=3D"">Note that other CPUs will not call rump_init(), they will =
 simply schedule some queued tasks.</span><br style=3D"caret-color: =
 rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
 normal; font-variant-caps: normal; font-weight: normal; letter-spacing: =
 normal; text-align: start; text-indent: 0px; text-transform: none; =
 white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""><span style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none; float: none; display: inline !important;" =
 class=3D"">Does it make sense?</span><br style=3D"caret-color: rgb(0, 0, =
 0); font-family: Helvetica; font-size: 12px; font-style: normal; =
 font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
 text-align: start; text-indent: 0px; text-transform: none; white-space: =
 normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
 text-decoration: none;" class=3D""></div></blockquote></div><br =
 class=3D""><div class=3D"">Yes, but where is rump_init_callback() called =
 with a non-NULL argument?</div><div class=3D""><br class=3D""></div><div =
 class=3D"">christos</div></body></html>=

 --Apple-Mail=_2278752C-A2D9-492F-92FE-09F0FF283529--

 --Apple-Mail=_5E240126-329A-4497-9786-4E4C75FE017D
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCX6LpogAKCRBxESqxbLM7
 Os2fAKCgthLY0UahAR0J7sQYWm4CnACUDwCghTMlGa0Ql8lXhmXBH8W72kXUWw4=
 =tiET
 -----END PGP SIGNATURE-----

 --Apple-Mail=_5E240126-329A-4497-9786-4E4C75FE017D--

From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Wed, 4 Nov 2020 13:47:21 -0500

 This is called *in lieu* of rump_init() for > 1 CPU (for SMP when a 
 callback function to wake up sleeping CPUs is needed).

 Basically, it should be new rump_init() but there is plenty of code 
 which calls it in the current source tree

 On 11/4/20 12:49 PM, Christos Zoulas wrote:
 >
 >
 >> On Nov 4, 2020, at 12:42 PM, Ruslan Nikolaev <nruslan_devel@yahoo.com 
 >> <mailto:nruslan_devel@yahoo.com>> wrote:
 >>
 >> No, Christos, the point is different here.
 >>
 >> Only the bootstrap CPU calls rump_init() even after our change.
 >>
 >> However, during rump_init() other CPUs are already scheduled some 
 >> tasks, so we need to wake them up, we cannot simply wait until 
 >> rump_init() finishes. If we wake them too early (i.e., before 
 >> rump_init()), their state will not be properly initialized yet. So, 
 >> we need to wake them up while in rump_init() but only *after* their 
 >> state is fully initialized.
 >>
 >> So, this is when the bootstrap CPU will call the callback function.
 >>
 >>
 >> Note that other CPUs will not call rump_init(), they will simply 
 >> schedule some queued tasks.
 >>
 >>
 >> Does it make sense?
 >
 > Yes, but where is rump_init_callback() called with a non-NULL argument?
 >
 > christos

From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55781 CVS commit: src/sys/rump
Date: Wed, 4 Nov 2020 17:06:39 -0500

 Module Name:	src
 Committed By:	christos
 Date:		Wed Nov  4 22:06:39 UTC 2020

 Modified Files:
 	src/sys/rump/include/rump: rump.h
 	src/sys/rump/librump/rumpkern: rump.c

 Log Message:
 PR/55781: Ruslan Nikolaev: rump_init() does differentiate when all CPUs are
 initialized


 To generate a diff of this commit:
 cvs rdiff -u -r1.72 -r1.73 src/sys/rump/include/rump/rump.h
 cvs rdiff -u -r1.349 -r1.350 src/sys/rump/librump/rumpkern/rump.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Ruslan Nikolaev <nruslan_devel@yahoo.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/55781: more rump fixes
Date: Thu, 5 Nov 2020 23:21:46 -0500

 Thanks, Christos for merging this change. I also updated kern/55777 with 
 a new patch. I have also created kern/55789 for rump drivers patches.

 On 11/4/20 12:49 PM, Christos Zoulas wrote:
 >
 > Yes, but where is rump_init_callback() called with a non-NULL argument?
 >
 > christos

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.