NetBSD Problem Report #17180

Received: (qmail 18257 invoked by uid 605); 6 Jun 2002 06:13:07 -0000
Message-Id: <200206060612.g566CjX28764@polka.clank.org>
Date: Thu, 6 Jun 2002 01:12:47 -0500 (CDT)
From: Scott Reynolds <scottr@clank.org>
Sender: gnats-bugs-owner@netbsd.org
Reply-To: scottr@clank.org
To: gnats-bugs@gnats.netbsd.org
Subject: m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
X-Send-Pr-Version: 3.95

>Number:         17180
>Category:       port-m68k
>Synopsis:       m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    scottr
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jun 06 06:14:01 +0000 2002
>Closed-Date:    Fri Jul 05 02:55:54 +0000 2002
>Last-Modified:  Fri Jul 05 02:55:54 +0000 2002
>Originator:     Scott Reynolds
>Release:        NetBSD 1.6_BETA1 2002-06-03
>Organization:
>Environment:
System: NetBSD maverick 1.6_BETA1 NetBSD 1.6_BETA1 (GENERIC) #0: Tue Jun 4 04:45:22 UTC 2002 autobuild@tgm.daemon.org:/autobuild/mac68k/OBJ/autobuild/src/sys/arch/mac68k/compile/GENERIC mac68k
Architecture: m68k
Machine: mac68k
>Description:
	The bus_dmamap_sync() implementation in sys/arch/m68k/m68k/bus_dma.c
	assumes the general availability of DCFL() and DCFP() to flush lines
	and pages in the data cache. Because there is no directly corres-
	onding instruction for the '020 and '030, however, compiling without
	'040 and '060 support fails. Additionally, in the case of a GENERIC
	kernel the default action for these two operations is to execute an
	'040-specific instruction. Attempting a DCFL() or DCFP() will at
	best cause an illegal instruction trap on the older processors.

	While the problem was observed on mac68k specifically, its nature
	suggests that it is likely to impact most m68k ports to some degree.

>How-To-Repeat:
	Configure e.g. a mac68k kernel without support for the 68040.
	Attempt to compile the kernel; it will fail while attempting to
	compile bus_dma.c.

		-or-

	Attempt to boot a GENERIC mac68k kernel on an '020 or '030.

>Fix:
	The brutal hack is to simply bracket all cases of DCFL() and DCFP()
	with

		#if defined(M68040) || defined(M68060)
			...
		#endif

	as in other sections that use it. One could instead argue that the
	default behavior of these operations should be a no-op, which would
	of course work around the problem. However, either of these work-		arounds is almost certainly not correct. There are likely some '030
	machines that need (or will in the future need) bus_dmamap_sync()
	to operate properly.

	One solution would be to implement a data cache flush operation for
	the 68030, then update bus_dmamap_sync() to use it as appropriate
	and skip the data cache flush on the 68020 altogether.
>Release-Note:
>Audit-Trail:

From: Steve Woodford <scw@netbsd.org>
To: Scott Reynolds <scottr@clank.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption
 regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 08:38:19 +0100 (BST)

 On Thu, 6 Jun 2002, Scott Reynolds wrote:

 > 	While the problem was observed on mac68k specifically, its nature
 > 	suggests that it is likely to impact most m68k ports to some degree.

 Well, an easy fix would be to do roughly what mvme68k does. That is, have
 two functions, one for 020/030 and one for 040/060. While the former
 doesn't actually do anything at this time, this could easily be changed
 should the need arise, as you said.

 The decision about which routine to use is made at runtime when the cpu
 type is known.

 Cheers, Steve


From: Frederick Bruckman <fredb@immanent.net>
To: Scott Reynolds <scottr@clank.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption
 regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 05:54:47 -0500 (CDT)

 On Thu, 6 Jun 2002, Scott Reynolds wrote:

 > 	The brutal hack is to simply bracket all cases of DCFL() and DCFP()
 > 	with
 >
 > 		#if defined(M68040) || defined(M68060)
 > 			...
 > 		#endif
 >
 > 	as in other sections that use it.

 But M68040 may be defined, even if you don't have an M68040 (MULTI_CPU
 case), so that would still have the same problem that's in _DCFL() and
 friends now.

 > One could instead argue that the
 > 	default behavior of these operations should be a no-op, which would
 > 	of course work around the problem.

 I posted a patch to do the later in

 	http://mail-index.netbsd.org/port-m68k/2002/06/01/0000.html

 A kernel built with this was reported to work fine on a PowerBook 145.

 > However, either of these work-
 >	arounds is almost certainly not correct. There are likely some '030
 > 	machines that need (or will in the future need) bus_dmamap_sync()
 > 	to operate properly.

 Hmm... are there any 68030 machines that do DMA?

 > 	One solution would be to implement a data cache flush operation for
 > 	the 68030, then update bus_dmamap_sync() to use it as appropriate
 > 	and skip the data cache flush on the 68020 altogether.

 My old Motorola assembler book says the "new" 68030 will have a
 write-through data cache, which therefore wouldn't ever need to be
 flushed, but I haven't checked the Motorola docs yet.

 Frederick


From: Scott Reynolds <scottr@clank.org>
To: Frederick Bruckman <fredb@immanent.net>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 09:13:39 -0500

 On Thursday, June 6, 2002, at 05:54  AM, Frederick Bruckman wrote:

 > But M68040 may be defined, even if you don't have an M68040 (MULTI_CPU
 > case), so that would still have the same problem that's in _DCFL() and
 > friends now.

 I don't follow you here; I can't see how M68040 would be defined unless 
 we've asked for it. In any event, it's incorrect to even call 
 DCFL()/DCFP() on a 68020 or 68030. These processors have no such 
 operation (data cache flush line/page).

 > Hmm... are there any 68030 machines that do DMA?

 Sure. There's even a Mac -- the IIfx.

 > My old Motorola assembler book says the "new" 68030 will have a
 > write-through data cache, which therefore wouldn't ever need to be
 > flushed, but I haven't checked the Motorola docs yet.

 In the context of DMA, the write-through cache doesn't help you. The 
 68030's data cache will need to be flushed in its entirety to ensure 
 coherency. It's not the write operation that's the problem here -- it's 
 a read from the same location after a DMA transfer.

 --scott


From: Scott Reynolds <scottr@clank.org>
To: Steve Woodford <scw@netbsd.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 09:16:54 -0500

 On Thursday, June 6, 2002, at 02:38  AM, Steve Woodford wrote:

 > Well, an easy fix would be to do roughly what mvme68k does.

 The mvme68k version of bus_dmamap_sync() appears to use the 040-specific 
 versions of DCFL()/DCFP() -- if there is an allowance for the earlier 
 processors, it's not obvious from the code.

 --scott


From: Steve Woodford <scw@netbsd.org>
To: Scott Reynolds <scottr@clank.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption
 regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 15:28:28 +0100 (BST)

 On Thu, 6 Jun 2002, Scott Reynolds wrote:

 > On Thursday, June 6, 2002, at 02:38  AM, Steve Woodford wrote:
 >
 > > Well, an easy fix would be to do roughly what mvme68k does.
 >
 > The mvme68k version of bus_dmamap_sync() appears to use the 040-specific
 > versions of DCFL()/DCFP() -- if there is an allowance for the earlier
 > processors, it's not obvious from the code.

 Ah, but it is. :)

 There are two functions:

 	bus_dmamap_sync_030()
 	bus_dmamap_sync_0460()

 The "bus_dmamap_sync" function pointer in the dma tag is initialised to
 one of the above at boot time according to which cpu is detected.

 DCFL_40()/DCFP_40() works for both 040 and 060 cpus, and avoids a function
 call in the case where support for both those cpus is compiled in.

 Cheers, Steve



From: Frederick Bruckman <fredb@immanent.net>
To: Scott Reynolds <scottr@clank.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption
 regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 11:32:21 -0500 (CDT)

 On Thu, 6 Jun 2002, Scott Reynolds wrote:

 > On Thursday, June 6, 2002, at 05:54  AM, Frederick Bruckman wrote:
 >
 > > But M68040 may be defined, even if you don't have an M68040 (MULTI_CPU
 > > case), so that would still have the same problem that's in _DCFL() and
 > > friends now.
 >
 > I don't follow you here; I can't see how M68040 would be defined unless
 > we've asked for it. In any event, it's incorrect to even call
 > DCFL()/DCFP() on a 68020 or 68030. These processors have no such
 > operation (data cache flush line/page).

 The typical GENERIC kernel defines all of M68040, M68030 and M68020.
 So, to adequetely cover the MULTI_CPU case, if you don't use the
 *run-time* selector in _DCFL(), then you'd have to use a run-time
 selector everywhere. Moreover, simply calling the macros in cacheops.h
 everywhere, no matter the processor, gets you an optimization in the
 M68030 and M68020 only case, as it avoids a switch.

 > > Hmm... are there any 68030 machines that do DMA?
 >
 > Sure. There's even a Mac -- the IIfx.

 OK.

 > > My old Motorola assembler book says the "new" 68030 will have a
 > > write-through data cache, which therefore wouldn't ever need to be
 > > flushed, but I haven't checked the Motorola docs yet.
 >
 > In the context of DMA, the write-through cache doesn't help you. The
 > 68030's data cache will need to be flushed in its entirety to ensure
 > coherency. It's not the write operation that's the problem here -- it's
 > a read from the same location after a DMA transfer.

 Right. But that only calls for a purge/invalidate. bus_dmasync_sync()
 looks like it follows a flush with a purge anyhow, so it's only needed
 to fill in the DCP?_30() functions? There has to be an application
 note somewhere that explains how you're supposed to do that.

 Frederick


From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: scottr@clank.org
Cc: gnats-bugs@gnats.netbsd.org, briggs@netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty
 assumption regarding DCFL()/DCFP()
Date: Thu, 6 Jun 2002 21:49:31 +0200

 [cc'ing Allen who integrated the m68k bus_dma stuff]

 At 1:12 Uhr -0500 6.6.2002, Scott Reynolds wrote:
 >	The bus_dmamap_sync() implementation in sys/arch/m68k/m68k/bus_dma.c
 >	assumes the general availability of DCFL() and DCFP() to flush lines
 >	and pages in the data cache. Because there is no directly corres-
 >	onding instruction for the '020 and '030, however, compiling without
 >	'040 and '060 support fails. Additionally, in the case of a GENERIC
 >	kernel the default action for these two operations is to execute an
 >	'040-specific instruction. Attempting a DCFL() or DCFP() will at
 >	best cause an illegal instruction trap on the older processors.
 >
 >	While the problem was observed on mac68k specifically, its nature
 >	suggests that it is likely to impact most m68k ports to some degree.

 I originally started with the next68k code and pointed Allen to it; the
 x68k equivalent that I came across later looks a bit tidier to me (not that
 I'd pretend to really understand the details ;).

 Look in x68k/x68k/bus.c, probably the implementation of
 x68k_bus_dmamap_sync() will do what you want.

 	hauke


 --
 /~\  The ASCII Ribbon Campaign       "They that can give up essential liberty
 \ /    No HTML/RTF in email          to obtain a little temporary safety
  X     No Word docs in email         deserve neither liberty nor safety."
 / \  Respect for open standards                    -- Benjamin Franklin, 1759



From: Scott Reynolds <scottr@netbsd.org>
To: Steve Woodford <scw@netbsd.org>
Cc: gnats-bugs@gnats.netbsd.org, briggs@netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
Date: Mon, 10 Jun 2002 13:51:23 -0500

 Sorry, yes, of course mvme68k has split the functions.

 It seems to me that this should be pulled into the generic m68k 
 bus_dma.c somehow. I'm torn between splitting bus_dmamap_sync() into 
 separate functions and using a switch statement (on cputype, as 
 cacheops.c does). In the interest of keeping the interface clean and 
 efficient I'm tending towards the latter. The CPU-specific parts would 
 of course be bracketed with the appropriate #if conditionals.

 --scott


From: Scott Reynolds <scottr@clank.org>
To: Frederick Bruckman <fredb@immanent.net>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
Date: Mon, 10 Jun 2002 17:42:37 -0500

 On Thursday, June 6, 2002, at 11:32 AM, Frederick Bruckman wrote:

 > The typical GENERIC kernel defines all of M68040, M68030 and M68020.
 > So, to adequetely cover the MULTI_CPU case, if you don't use the
 > *run-time* selector in _DCFL(), then you'd have to use a run-time
 > selector everywhere.

 This is true, but see below.

 > Moreover, simply calling the macros in cacheops.h
 > everywhere, no matter the processor, gets you an optimization in the
 > M68030 and M68020 only case, as it avoids a switch.

 The point is that the operations don't exists on the 020/030, so 
 defining them to be a no-op is incorrect. This PR specifically points 
 out the faulty assumptions that these operations exist on processors 
 other than the 040/060. The data cache on the 68030 is handled 
 differently and bus_dmamap_sync() should reflect this difference.

 A switch organized in the following way will optimize very well, 
 removing all of the code related to the switch statement if you only 
 define a single processor:

 switch (var) {
 default:
 #ifdef M68020
 case CPU_68020:
 	foo;
 	break;
 #endif
 #ifdef M68030
 case CPU_68030:
 	bar;
 	break;
 #endif
 /* etc. */
 }

 This is how cacheops.c manages to balance general functionality with 
 efficiency. If you only define e.g. M68030, the entire switch statement 
 itself is reduced to "bar" -- that is, whatever code is executed for 
 that case. In my opinion, this is a good example of leveraging the 
 optimizer to get the desired result while maintaining the readability of 
 the code.

 > [re: write-through cache - sr] But that only calls for a 
 > purge/invalidate. bus_dmasync_sync()
 > looks like it follows a flush with a purge anyhow,

 Point taken, good catch.

 > so it's only needed
 > to fill in the DCP?_30() functions? There has to be an application
 > note somewhere that explains how you're supposed to do that.

 There are two operations -- "clear" (really, invalidate) the entire 
 cache or a single entry. There are no corresponding page or line 
 operations, which complicates things a bit. In any event it's probably 
 most efficient to simply do a DCPA_30() in the 030 case rather than 
 waste cycles clearing invidual entries in a 256-byte data cache.

 --scott

 PS - Actually implementing DCPA_30() is a different issue than the one 
 pointed out here, and is not directly related to the PR. I think it's 
 sufficient to call it as a no-op as it currently exists to close the PR. 
 This removes the 040 instruction from the code path which is currently 
 bombing the 020/030 machines.


From: Scott Reynolds <scottr@clank.org>
To: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Cc: gnats-bugs@gnats.netbsd.org, briggs@netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption regarding DCFL()/DCFP()
Date: Mon, 10 Jun 2002 17:53:38 -0500

 On Thursday, June 6, 2002, at 02:49 PM, Hauke Fath wrote:

 > Look in x68k/x68k/bus.c, probably the implementation of
 > x68k_bus_dmamap_sync() will do what you want.

 It looks like it will solve the problem, but I'm concerned that the 
 instruction cache is unnecessarily invalidated here. Also, it seems like 
 it could be cleaned up a bit -- the non-040/060 case is essentially 
 duplicated. It's certainly something to reference for a solution.

 --scott


From: Frederick Bruckman <fredb@immanent.net>
To: Scott Reynolds <scottr@clank.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-m68k/17180: m68k bus_dmamap_sync() makes faulty assumption
 regarding DCFL()/DCFP()
Date: Mon, 10 Jun 2002 23:35:51 -0500 (CDT)

 On Mon, 10 Jun 2002, Scott Reynolds wrote:

 > On Thursday, June 6, 2002, at 11:32 AM, Frederick Bruckman wrote:
 >
 > A switch organized in the following way will optimize very well,
 > removing all of the code related to the switch statement if you only
 > define a single processor:
 >
 > switch (var) {
 > default:
 > #ifdef M68020
 > case CPU_68020:
 > 	foo;
 > 	break;
 > #endif
 > #ifdef M68030
 > case CPU_68030:
 > 	bar;
 > 	break;
 > #endif
 > /* etc. */
 > }
 >
 > This is how cacheops.c manages to balance general functionality with
 > efficiency. If you only define e.g. M68030, the entire switch statement
 > itself is reduced to "bar" -- that is, whatever code is executed for
 > that case. In my opinion, this is a good example of leveraging the
 > optimizer to get the desired result while maintaining the readability of
 > the code.

 Yes, I see how that works. Take note: currently, all the cases in all
 operations in cacheops.c are not completely filled in, so (unless my
 understanding is faulty) having "default:" at the top sometimes drops
 you in the case for the wrong processor. That, I believe, is the cause
 of the immediate faults on the 68030 powerbooks (though I can't
 explain why floating point exceptions, and not illegal instruction
 errors).

 Filling in all cases would make the "default:" thing moot. I've
 come to think now, that even after filling them all in, it would
 be dangerous to simply leave the default case at the top. Rather, an
 unknown processor should probably cause a panic, because, while it's
 true that the minimal, 68020 handling is not likely to cause an
 immediate fault on a newer processor, it would lead to undefined
 behaviour, if you ever really needed to flush that cache.

 BTW, if you define *only* M68030, you don't use the functions in
 cacheops.c, but rather only the macros in cacheops_30.h, so there's no
 need or opportunity for the compiler to optimize away the switch. But
 you're probably right that it would; I hadn't thought about that. I
 guess the linker is not trusted to optimize away the do-nothing
 function calls (even though a smart compiler might), and that's why
 the present system, with some of the macros in cacheops_?0.h defining
 away to nothing.

 > > so it's only needed
 > > to fill in the DCP?_30() functions? There has to be an application
 > > note somewhere that explains how you're supposed to do that.
 >
 > There are two operations -- "clear" (really, invalidate) the entire
 > cache or a single entry. There are no corresponding page or line
 > operations, which complicates things a bit. In any event it's probably
 > most efficient to simply do a DCPA_30() in the 030 case rather than
 > waste cycles clearing invidual entries in a 256-byte data cache.

 That makes sense.

 > PS - Actually implementing DCPA_30() is a different issue than the one
 > pointed out here, and is not directly related to the PR. I think it's
 > sufficient to call it as a no-op as it currently exists to close the PR.
 > This removes the 040 instruction from the code path which is currently
 > bombing the 020/030 machines.

 On an M68030, trap #12 ends up calling DCIU_30(). Perhaps that would
 be the one to implement (as you described, clearing the whole thing)?

 Frederick

State-Changed-From-To: open->analyzed 
State-Changed-By: scottr 
State-Changed-When: Tue Jun 25 15:46:52 PDT 2002 
State-Changed-Why:  
The discussion here is a reasonably complete analysis of the problem. 
I've put together a patch (also sent to Allen) to implement the switch 
on cputype in bus_dmamap_sync(). 
State-Changed-From-To: analyzed->closed 
State-Changed-By: scottr 
State-Changed-When: Thu Jul 4 19:55:05 PDT 2002 
State-Changed-Why:  
Fixed in rev 1.3 of bus_dma.c, checked in earlier today. 


Responsible-Changed-From-To: port-m68k-maintainer->scottr 
Responsible-Changed-By: scottr 
Responsible-Changed-When: Thu Jul 4 19:55:05 PDT 2002 
Responsible-Changed-Why:  
>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.