NetBSD Problem Report #54994
From www@netbsd.org Fri Feb 21 11:14:14 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 4E32C1A9213
for <gnats-bugs@gnats.NetBSD.org>; Fri, 21 Feb 2020 11:14:14 +0000 (UTC)
Message-Id: <20200221111413.717EA1A9217@mollari.NetBSD.org>
Date: Fri, 21 Feb 2020 11:14:13 +0000 (UTC)
From: rokuyama.rk@gmail.com
Reply-To: rokuyama.rk@gmail.com
To: gnats-bugs@NetBSD.org
Subject: Critical bug in uarea_poolpage_alloc() for archs with __HAVE_CPU_UAREA_ROUTINES
X-Send-Pr-Version: www-1.0
>Number: 54994
>Category: kern
>Synopsis: Critical bug in uarea_poolpage_alloc() for archs with __HAVE_CPU_UAREA_ROUTINES
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Fri Feb 21 11:15:00 +0000 2020
>Last-Modified: Thu Mar 05 12:25:01 +0000 2020
>Originator: Rin Okuyama
>Release: 9.99.47 and netbsd-9 at least
>Organization:
Department of Physics, Meiji University
>Environment:
NetBSD obs266 9.99.47 NetBSD 9.99.47 (OBS266) #154: Thu Feb 20 17:28:10 JST 2020 rin@latipes:/build/src/sys/arch/evbppc/compile/OBS266 evbppc
>Description:
For archs with __HAVE_CPU_UAREA_ROUTINES, i.e., alpha, mips, powerpc,
and riscv, uarea_poolpage_alloc() falls back to uvm_km_alloc() if
cpu_uarea_alloc() fails:
https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c#uarea_poolpage_alloc
This behavior is incorrect.
For these archs, cpu_uarea_alloc() allocates direct-mapped physically
contignous pages for u-area by using uvm_pglistalloc():
https://nxr.netbsd.org/source/s?refs=cpu_uarea_alloc&project=src
Here, uvm_pglistalloc() may give up allocating memory even if waitok:
https://netbsd.gw.com/cgi-bin/man-cgi?uvm_pglistalloc++NetBSD-current
This is because there's no way to wake up a process when contignous
pages become available. In this case, cpu_uarea_alloc() returns NULL,
and uvm_km_alloc() is used instead. However, pages allocated by
uvm_km_alloc() are not physically contignous, of course.
Since low-level routines for these archs assume physically contignous
u-area, this results in a catastrophe; pages happens to follow the 1st
page of u-area will be randomly overwritten.
I found this is the cause of random kernel crashes on an ibm4xx box
under heavy load, which has small memory and large page size.
>How-To-Repeat:
Fork a lot of processes/threads on above mentioned archs. For my
ibm4xx box with 128MB memory, /usr/tests/lib/librumpclient/h_execthr
causes kernel crash almost always in multi-user mode.
>Fix:
Stop uarea_poolpage_alloc() to fall back into uvm_km_alloc(), and let
it return NULL if cpu_uarea_alloc() fails. Then, fork(2) fails with
ENOMEM correctly when contignous pages are not available, instead of
mysterious kernel crashes.
However, there still remains a minor problem. Since u-area is allocated
with PR_WAITOK,
https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c#uvm_uarea_alloc
pool subsystem assumes that uarea_poolpage_alloc() never return NULL.
KASSERT failures take place otherwise. We therefore need to add a new
flag, PR_MAYFAIL for example, to indicate that pa_alloc() may fail
even for PR_WAITOK.
Here's patch for uvm as well as pool subsystem:
http://www.netbsd.org/~rin/uarea_poolpage_20200221.patch
Note that for ibm4xx, we need only single page for u-area; PAGE_SIZE =
16KB is the same size of u-area for other powerpc processors. I will
change UPAGES from 2 to 1 for ibm4xx as a workaround, but this problem
still needs to be fixed at a fundamental level.
>Audit-Trail:
From: "Rin Okuyama" <rin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54994 CVS commit: src/sys/arch/powerpc/include
Date: Fri, 21 Feb 2020 12:56:37 +0000
Module Name: src
Committed By: rin
Date: Fri Feb 21 12:56:37 UTC 2020
Modified Files:
src/sys/arch/powerpc/include: param.h
Log Message:
Reduce UPAGES from 2 to 1 for ibm4xx, which was originally 1 and bumped
to 2 in rev 1.29:
http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/arch/powerpc/include/param.h#rev1.29
Since page size is 16KB on ibm4xx, USPACE is still 16KB, which is the
same size as on other powerpc processors.
This avoids kernel crash described in PR kern/54994. Also, even if the
PR is resolved, fork(2) easily fails with ENOMEM if UPAGES is 2, which
requires physically contiguous pages for u-area. No bad side effect is
observed as far as I can see.
XXX
Even with this commit, kern/54994 still critically affects other archs
with __HAVE_CPU_UAREA_ROUTINES, i.e., alpha, mips, powerpc/{oea,booke},
and riscv.
To generate a diff of this commit:
cvs rdiff -u -r1.31 -r1.32 src/sys/arch/powerpc/include/param.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Jason Thorpe <thorpej@me.com>
To: rin@netbsd.org
Cc: kern-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Fri, 21 Feb 2020 07:52:45 -0800
> On Feb 21, 2020, at 3:15 AM, rokuyama.rk@gmail.com wrote:
>=20
> Since low-level routines for these archs assume physically contignous
> u-area, this results in a catastrophe; pages happens to follow the 1st
> page of u-area will be randomly overwritten.
I don't have an Alpha handy to run your test case, but I don't think =
this is catastrophic on all platforms. As far as I can recall, the =
Alpha does not actually require a physically contiguous u-area. MIPS is =
the same in this regard (although it probably need to make sure it keeps =
UPAGES wired TLB entries for the u-area when the direct-mapping isn't =
used ... it certainly used to do that). The comment in the Alpha's =
cpu_uarea_alloc() is merely there to call out the requirement in order =
to be able to use the direct-mapped region.
Oh, turns out MIPS already has much more complex rules for when the =
direct-mapped region is used, so I suspect it's fine as well. I haven't =
audited the rest.
In any case, the short answer is: Only an issue if the PA of the u-area =
is provided to the CPU, not an issue if it does everything with the VA =
of the u-area.
-- thorpej
From: Nick Hudson <nick.hudson@gmx.co.uk>
To: Jason Thorpe <thorpej@me.com>, rin@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Fri, 21 Feb 2020 16:19:35 +0000
On 21/02/2020 15:52, Jason Thorpe wrote:
> Oh, turns out MIPS already has much more complex rules for when the dire=
ct-mapped region is used, so I suspect it's fine as well. I haven't audit=
ed the rest.
E&OE, but hopefully I fixed it all or marked it as not supported
appropriately.
I'd look again, but remembering mips isn't a priority for me.
Nick
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Nick Hudson <nick.hudson@gmx.co.uk>, Jason Thorpe <thorpej@me.com>
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Mon, 24 Feb 2020 09:29:58 +0900
Jason, Nick, thank you for your kind comments!
__HAVE_CPU_UAREA_ROUTINES is enabled for alpha, mips,
powerpc/{oae,ibm4xx,booke}, and riscv. I investigated whether it is
really necessary or not for these archs (except for riscv).
In short, most of these archs do *not* need direct-mapped physically
contiguous u-area for now, as far as I can see (source code reading
and experiment on powerpc/oea, just experiment on other archs). Only
the exception is powerpc/ibm4xx, which should also be fixed.
So is it time to retire __HAVE_CPU_UAREA_ROUTINES?
Details:
(1) powerpc
For powerpc architectures, MMU is turned off by hardware when exception
occurs. We need direct-mapped physically contiguous u-area, if we
manipulate kernel stack in trap handlers, before MMU is reenabled.
For powerpc/oea, external interruption handler used to do that. But it
was rewritten to manipulate stack after MMU is enabled, from
powerpc/trap_subr.S rev 1.67:
http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/arch/powerpc/powerpc/trap_subr.S#rev1.67
Actually, kernel works fine on macppc (Mac mini G4) even if
__HAVE_CPU_UAREA_ROUTINES is manually turned off; boots multiuser, and
builds pkgsrc/lang/perl5 for example.
For powerpc/ibm4xx, trap handlers are designed based on old powerpc/oea
implementation. And interruption handler still manipulates kernel stack
with MMU disabled. I thought it can be fixed mechanically, but something
goes wrong with softint_dispatch(). I will examine further.
For powerpc/booke, I'm not familiar to its architecture nor organization
of kernel source codes. So I turned __HAVE_CPU_UAREA_ROUTINES off to see
what happens. RB800 (MPC8544E) also boots multiuser and builds lang/perl5.
(2) alpha, mips
I'm also not familiar to these archs, so I did experiment to turn off
__HAVE_CPU_UAREA_ROUTINES.
For alpha, DS10 (21264A) boots multiuser, and builds lang/perl5.
For mips, MobileGearII MC/R730 (VR4121; MIPS3 with ENABLE_MIPS_4KB_PAGE)
also boots multiuser. I am too lazy to build perl on this machine, but
hello world builds and runs at least :).
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Nick Hudson <nick.hudson@gmx.co.uk>, Jason Thorpe <thorpej@me.com>
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Tue, 25 Feb 2020 20:24:18 +0900
On 2020/02/24 9:29, Rin Okuyama wrote:
> __HAVE_CPU_UAREA_ROUTINES is enabled for alpha, mips,
> powerpc/{oae,ibm4xx,booke}, and riscv. I investigated whether it is
> really necessary or not for these archs (except for riscv).
>
> In short, most of these archs do *not* need direct-mapped physically
> contiguous u-area for now, as far as I can see (source code reading
> and experiment on powerpc/oea, just experiment on other archs). Only
> the exception is powerpc/ibm4xx, which should also be fixed.
>
> So is it time to retire __HAVE_CPU_UAREA_ROUTINES?
Oops, mips64 depends on direct-mapped u-area (with UPAGES == 1);
KASSERT fires on ERLITE (MIPS64_OCTEON) if __HAVE_CPU_UAREA_ROUTINES
is turned off:
https://nxr.netbsd.org/xref/src/sys/arch/mips/mips/vm_machdep.c#136
117 #if (USPACE > PAGE_SIZE) || !defined(_LP64)
...
135 #else
136 KASSERT(pmap_md_direct_mapped_vaddr_p(ua2));
137 #endif
If this KASSERT is commented out, system hangs when exec /sbin/init,
and I cannot even enter ddb from console.
Thanks,
rin
From: Nick Hudson <nick.hudson@gmx.co.uk>
To: Rin Okuyama <rokuyama.rk@gmail.com>, Jason Thorpe <thorpej@me.com>
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Tue, 25 Feb 2020 12:38:44 +0000
On 25/02/2020 11:24, Rin Okuyama wrote:
> On 2020/02/24 9:29, Rin Okuyama wrote:
>> __HAVE_CPU_UAREA_ROUTINES is enabled for alpha, mips,
>> powerpc/{oae,ibm4xx,booke}, and riscv. I investigated whether it is
>> really necessary or not for these archs (except for riscv).
>>
>> In short, most of these archs do *not* need direct-mapped physically
>> contiguous u-area for now, as far as I can see (source code reading
>> and experiment on powerpc/oea, just experiment on other archs). Only
>> the exception is powerpc/ibm4xx, which should also be fixed.
>>
>> So is it time to retire __HAVE_CPU_UAREA_ROUTINES?
>
> Oops, mips64 depends on direct-mapped u-area (with UPAGES =3D=3D 1);
> KASSERT fires on ERLITE (MIPS64_OCTEON) if __HAVE_CPU_UAREA_ROUTINES
> is turned off:
On mips64 iirc it's architecturally defined to have direct-map,
i.e. XKPHYS, so the fallback from cpu_uarea_alloc to
uvm_km_alloc "should" never happen.
This conditional is also slightly off I think
http://src.illumos.org/source/xref/netbsd-src/sys/uvm/uvm_glue.c#247
247 while (USPACE =3D=3D PAGE_SIZE && USPACE_ALIGN =3D=3D 0) {
the following is more correct (imo)
while (USPACE =3D=3D PAGE_SIZE &&
(USPACE_ALIGN =3D=3D 0 || USPACE_ALIGN =3D=3D PAGE_SIZE)) {
All that said I'm facing my own issues with uvm_pglistalloc not waiting
for > PAGE_SIZE physically contiguous allocations (2 pages in my case)
Nick
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Nick Hudson <nick.hudson@gmx.co.uk>, Jason Thorpe <thorpej@me.com>
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Wed, 26 Feb 2020 23:17:44 +0900
Hi,
On 2020/02/25 21:38, Nick Hudson wrote:> On 25/02/2020 11:24, Rin Okuyama wrote:
>> On 2020/02/24 9:29, Rin Okuyama wrote:
>>> __HAVE_CPU_UAREA_ROUTINES is enabled for alpha, mips,
>>> powerpc/{oae,ibm4xx,booke}, and riscv. I investigated whether it is
>>> really necessary or not for these archs (except for riscv).
>>>
>>> In short, most of these archs do *not* need direct-mapped physically
>>> contiguous u-area for now, as far as I can see (source code reading
>>> and experiment on powerpc/oea, just experiment on other archs). Only
>>> the exception is powerpc/ibm4xx, which should also be fixed.
>>>
>>> So is it time to retire __HAVE_CPU_UAREA_ROUTINES?
>>
>> Oops, mips64 depends on direct-mapped u-area (with UPAGES == 1);
>> KASSERT fires on ERLITE (MIPS64_OCTEON) if __HAVE_CPU_UAREA_ROUTINES
>> is turned off:
>
> On mips64 iirc it's architecturally defined to have direct-map,
> i.e. XKPHYS, so the fallback from cpu_uarea_alloc to
> uvm_km_alloc "should" never happen.
>
> This conditional is also slightly off I think
>
> http://src.illumos.org/source/xref/netbsd-src/sys/uvm/uvm_glue.c#247
>
> 247 while (USPACE == PAGE_SIZE && USPACE_ALIGN == 0) {
>
> the following is more correct (imo)
>
> while (USPACE == PAGE_SIZE &&
> (USPACE_ALIGN == 0 || USPACE_ALIGN == PAGE_SIZE)) {
>
>
> All that said I'm facing my own issues with uvm_pglistalloc not waiting
> for > PAGE_SIZE physically contiguous allocations (2 pages in my case)
Yeah, I think you are correct. Memory allocated in this loop is page
itself, and therefore page-aligned. With this change, as well as
similar one in uarea_poolpage_free(),
https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c#277
277 if (USPACE == PAGE_SIZE && USPACE_ALIGN == 0) {
--->
if (USPACE == PAGE_SIZE &&
(USPACE_ALIGN == 0 || USPACE_ALIGN == PAGE_SIZE)) {
kernel on mips64eb works fine without __HAVE_CPU_UAREA_ROUTINES.
So, all ports including mips64 do not need CPU_UAREA_ROUTINES anymore,
as far as I can see. Can we try turning off __HAVE_CPU_UAREA_ROUTINES to
see what happens, or should we be more careful?
Thanks,
rin
From: Jason Thorpe <thorpej@me.com>
To: Rin Okuyama <rokuyama.rk@gmail.com>
Cc: Nick Hudson <nick.hudson@gmx.co.uk>,
kern-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Wed, 26 Feb 2020 06:32:23 -0800
> On Feb 26, 2020, at 6:17 AM, Rin Okuyama <rokuyama.rk@gmail.com> =
wrote:
>=20
> So, all ports including mips64 do not need CPU_UAREA_ROUTINES anymore,
> as far as I can see. Can we try turning off __HAVE_CPU_UAREA_ROUTINES =
to
> see what happens, or should we be more careful?
Platforms never "needed" __HAVE_CPU_UAREA_ROUTINES ... we didn't always =
have them.
I think we should keep them ... they are a useful optimization when they =
can do their work effectively. However, we need to ensure that when =
they can't meet their constraints that the fallback still works.
-- thorpej
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Jason Thorpe <thorpej@me.com>
Cc: Nick Hudson <nick.hudson@gmx.co.uk>, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Thu, 27 Feb 2020 00:13:08 +0900
On 2020/02/26 23:32, Jason Thorpe wrote:
>> On Feb 26, 2020, at 6:17 AM, Rin Okuyama <rokuyama.rk@gmail.com> wrote:
>>
>> So, all ports including mips64 do not need CPU_UAREA_ROUTINES anymore,
>> as far as I can see. Can we try turning off __HAVE_CPU_UAREA_ROUTINES to
>> see what happens, or should we be more careful?
>
> Platforms never "needed" __HAVE_CPU_UAREA_ROUTINES ... we didn't always have them.
>
> I think we should keep them ... they are a useful optimization when they can do their work effectively. However, we need to ensure that when they can't meet their constraints that the fallback still works.
Certainly. Then, what should we do?
Until now, we've learned:
(1) uarea_poolpage_alloc() can fall back into uvm_km_alloc():
https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c#269
This does not work if low-level routines need physically
contiguous (i.e., direct-mapped) pages for u-area.
(2) However, all ports with __HAVE_CPU_UAREA_ROUTINES actually do
*not* need contiguous u-area anymore, as far as we can see.
(3) Unfortunately, (2) does not mean that fallback of (1) is safe.
If some ports, that need direct-mapped u-area, bump USPACE from
1 to 2 (or more), fallback of uvm_km_alloc() results in memory
corruption. This is what we observed on powerpc/ibm4xx.
So, we have some options to do:
(a) Add MD flag to forbid fallback of uvm_km_alloc().
Or if this seems too much,
(b) Leave some comments in uarea_poolpage_alloc().
Thoughts?
Thanks,
rin
From: Jason Thorpe <thorpej@me.com>
To: Rin Okuyama <rokuyama.rk@gmail.com>
Cc: Nick Hudson <nick.hudson@gmx.co.uk>,
kern-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Wed, 26 Feb 2020 14:13:05 -0800
> On Feb 26, 2020, at 7:13 AM, Rin Okuyama <rokuyama.rk@gmail.com> =
wrote:
>=20
> Certainly. Then, what should we do?
>=20
> Until now, we've learned:
>=20
> (1) uarea_poolpage_alloc() can fall back into uvm_km_alloc():
>=20
> https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c#269
>=20
> This does not work if low-level routines need physically
> contiguous (i.e., direct-mapped) pages for u-area.
>=20
> (2) However, all ports with __HAVE_CPU_UAREA_ROUTINES actually do
> *not* need contiguous u-area anymore, as far as we can see.
AFAIK, they *never* did. Certainly, Alpha does not require a =
physically-contiguous u-area, neither does x86. Heck, neither does =
MIPS, assuming wired TLB entries are used to keep the kernel stack =
mapped. A physically contiguous u-area is ONLY required if you are =
using a direct-mapped segment to provide the address of the u-area to =
the CPU.
> (3) Unfortunately, (2) does not mean that fallback of (1) is safe.
> If some ports, that need direct-mapped u-area, bump USPACE from
> 1 to 2 (or more), fallback of uvm_km_alloc() results in memory
> corruption. This is what we observed on powerpc/ibm4xx.
>=20
> So, we have some options to do:
>=20
> (a) Add MD flag to forbid fallback of uvm_km_alloc().
>=20
> Or if this seems too much,
>=20
> (b) Leave some comments in uarea_poolpage_alloc().
>=20
> Thoughts?
We need to understand why the fallback fails on the platforms where it =
does fail. The following statements should all be true:
1- If physically-contiguous pages for the u-area can be allocated and =
mapped with a direct-mapped segment, we should be able to use that.
2- If phusically-contiguous pages for the u-area cannot be allocated, =
then the system should be able to use a u-area that is virtually mapped =
but not physically contiguous.
(2) used to be the way the system always worked for UPAGES > 1.
>=20
> Thanks,
> rin
-- thorpej
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Jason Thorpe <thorpej@me.com>
Cc: Nick Hudson <nick.hudson@gmx.co.uk>, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/54994: Critical bug in uarea_poolpage_alloc() for archs with
__HAVE_CPU_UAREA_ROUTINES
Date: Mon, 2 Mar 2020 09:03:39 +0900
On 2020/02/27 7:13, Jason Thorpe wrote:
>> On Feb 26, 2020, at 7:13 AM, Rin Okuyama <rokuyama.rk@gmail.com> wrote:
>>
>> Certainly. Then, what should we do?
>>
>> Until now, we've learned:
>>
>> (1) uarea_poolpage_alloc() can fall back into uvm_km_alloc():
>>
>> https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c#269
>>
>> This does not work if low-level routines need physically
>> contiguous (i.e., direct-mapped) pages for u-area.
>>
>> (2) However, all ports with __HAVE_CPU_UAREA_ROUTINES actually do
>> *not* need contiguous u-area anymore, as far as we can see.
>
> AFAIK, they *never* did. Certainly, Alpha does not require a physically-contiguous u-area, neither does x86. Heck, neither does MIPS, assuming wired TLB entries are used to keep the kernel stack mapped. A physically contiguous u-area is ONLY required if you are using a direct-mapped segment to provide the address of the u-area to the CPU.
OK
>> (3) Unfortunately, (2) does not mean that fallback of (1) is safe.
>> If some ports, that need direct-mapped u-area, bump USPACE from
>> 1 to 2 (or more), fallback of uvm_km_alloc() results in memory
>> corruption. This is what we observed on powerpc/ibm4xx.
>>
>> So, we have some options to do:
>>
>> (a) Add MD flag to forbid fallback of uvm_km_alloc().
>>
>> Or if this seems too much,
>>
>> (b) Leave some comments in uarea_poolpage_alloc().
>>
>> Thoughts?
>
> We need to understand why the fallback fails on the platforms where it does fail. The following statements should all be true:
>
> 1- If physically-contiguous pages for the u-area can be allocated and mapped with a direct-mapped segment, we should be able to use that.
>
> 2- If phusically-contiguous pages for the u-area cannot be allocated, then the system should be able to use a u-area that is virtually mapped but not physically contiguous.
>
> (2) used to be the way the system always worked for UPAGES > 1.
As far as I can see, all archs except for powerpc/ibm4xx satisfy both
(1) and (2). (More precisely, they seem not to requires direct-mapped
memory for u-area.)
For ibm4xx, the external interrupt handler uses kernel stack before
enabling translation by MMU:
https://nxr.netbsd.org/xref/src/sys/arch/powerpc/ibm4xx/trap_subr.S#INTR_SAVE
I managed to enable MMU before stack manipulation, but it causes kernel
panic due to TLB miss in the interrupt handler (see details below).
Thanks,
rin
Details:
By enabling MMU before using kernel stack in the interrupt handler,
kernel panic occurs when UPAGES == 2 and __HAVE_FAST_SOFTINTS. This is
due to TLB miss in the 2nd page of u-area.
However, I do not understand the situation yet; (a) why such a TLB miss
does not results in kernel panic for other exception handlers, that
already enable MMU in a similar manner:
https://nxr.netbsd.org/xref/src/sys/arch/powerpc/ibm4xx/trap_subr.S#FRAME_SETUP
and (b) why not without __HAVE_FAST_SOFTINTS.
Seems like a problem in __HAVE_FAST_SOFTINTS v.s. powerpc/ibm4xx.
This is not a very urgent matter, because there is no problem with
UPAGES == 1 for ibm4xx. But should get fixed, of course.
From: "Rin Okuyama" <rin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54994 CVS commit: src/sys/uvm
Date: Thu, 5 Mar 2020 12:21:00 +0000
Module Name: src
Committed By: rin
Date: Thu Mar 5 12:21:00 UTC 2020
Modified Files:
src/sys/uvm: uvm_glue.c
Log Message:
Part of PR kern/54994:
Memory allocated in the fast path of uarea_poolpage_alloc() is
a page itself. Therefore, it is obviously page-aligned.
Pointed out by skrll.
To generate a diff of this commit:
cvs rdiff -u -r1.176 -r1.177 src/sys/uvm/uvm_glue.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home