NetBSD Problem Report #42212

From Wolfgang.Stukenbrock@nagler-company.com  Thu Oct 22 16:45:44 2009
Return-Path: <Wolfgang.Stukenbrock@nagler-company.com>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 5B43863B877
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 22 Oct 2009 16:45:44 +0000 (UTC)
Message-Id: <20091022164541.614794EA9FE@s012.nagler-company.com>
Date: Thu, 22 Oct 2009 18:45:41 +0200 (CEST)
From: Wolfgang.Stukenbrock@nagler-company.com
Reply-To: Wolfgang.Stukenbrock@nagler-company.com
To: gnats-bugs@gnats.NetBSD.org
Subject: dma memory allocation broken on x86 architectures
X-Send-Pr-Version: 3.95

>Number:         42212
>Category:       port-amd64
>Synopsis:       dma memory allocation broken on x86 architectures
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    port-amd64-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 22 16:50:00 +0000 2009
>Originator:     Wolfgang Stukenbrock
>Release:        NetBSD 4.0
>Organization:
Dr. Nagler & Company GmbH

>Environment:


System: NetBSD s012 4.0 NetBSD 4.0 (NSW-S012) #9: Fri Mar 13 12:31:52 CET 2009 wgstuken@s012:/usr/src/sys/arch/amd64/compile/NSW-S012 amd64
Architecture: x86_64
Machine: amd64
>Description:
	The dma memory setup on X86 architecturs need to work around several "stupid" hardware, that
	cannot use DMA without restrictions. One common problem is, that DMA sections may not cross
	some address boundaries - e.g. 64k for IDE controlers.
	In the default setup of NetBSD MAXPHYS is set to 64k, so at least on these controlers the bug
	is not seen.
	Now the pciide-driver tries to setup DMA only with memory above 16M.
	In this case, the DMA stuff allocates a cookie-buffer and this allocation will fail in some situations.
	The reason for the problem is the fact, that the uvm_physmem allocation routine tries to allocate
	everything as continuous memory. It checks that the size is largen than the boundary restriction and
	returns EINVAL.
	This behaviour seems to be correct and should not be changed.
	The calling part may not allocate more than boundary bytes, because it only gets a list of pages back.
	In order to enforce the boundary restrictions, a new segment must be started if a boundary is crossed.
	Therefore it is much better to allocate chunks with at most boundary bytes until the requested
	size is allocated.

	remark: not only the amd64 port is affected. All x86 ports that doesn't have an own routine for this
		share this problem and are affected by the patch below.
		I haven't checked the dmamem-alloc routines for the other ports. Perhaps there is the same problem.
>How-To-Repeat:
	Set MAXPHYS to e.g. 1MB in the kernel config (remember to set MAXBSIZE to 64k - or you have to
	change some other source files too ..).
	Compile and run this kernel on a system with a pciide controler with some disks or cdrom attached
	to it. It will fail to allocate the DMA and revert to PIO mode..
	If you have an satalink controler with revision 1 (that has a 8k DMA boundary restriction), you
	can see this problem without increasing MAXPHYS.
	Perhaps there are some other controlers - not only IDE - too. I haven't checked this.
>Fix:
	The following patch to /usr/src/sys/arch/x86/x86/bus_dma.c will fix the problem.
	If there is a boundary restriction and the requested size is larger as the
	boundary, the allocation will be done in chunks up to at most boundary bytes.


--- bus_dma.c	2009/10/21 09:36:23	1.2
+++ bus_dma.c	2009/10/22 16:13:28
@@ -156,18 +156,15 @@
  * Allocate physical memory from the given physical address range.
  * Called by DMA-safe memory allocation methods.
  */
-int
-_bus_dmamem_alloc_range(bus_dma_tag_t t, bus_size_t size,
+static int
+__alloc_next_chunk(bus_size_t size,
     bus_size_t alignment, bus_size_t boundary, bus_dma_segment_t *segs,
-    int nsegs, int *rsegs, int flags, bus_addr_t low, bus_addr_t high)
+    int nsegs, int *curseg, int flags, bus_addr_t low, bus_addr_t high)
 {
 	paddr_t curaddr, lastaddr;
 	struct vm_page *m;
 	struct pglist mlist;
-	int curseg, error;
-
-	/* Always round the size. */
-	size = round_page(size);
+	int error;

 	/*
 	 * Allocate pages from the VM system.
@@ -182,9 +179,8 @@
 	 * returned by the VM code.
 	 */
 	m = mlist.tqh_first;
-	curseg = 0;
-	lastaddr = segs[curseg].ds_addr = VM_PAGE_TO_PHYS(m);
-	segs[curseg].ds_len = PAGE_SIZE;
+	lastaddr = segs[*curseg].ds_addr = VM_PAGE_TO_PHYS(m);
+	segs[*curseg].ds_len = PAGE_SIZE;
 	m = m->pageq.tqe_next;

 	for (; m != NULL; m = m->pageq.tqe_next) {
@@ -197,17 +193,57 @@
 		}
 #endif
 		if (curaddr == (lastaddr + PAGE_SIZE))
-			segs[curseg].ds_len += PAGE_SIZE;
+			segs[*curseg].ds_len += PAGE_SIZE;
 		else {
-			curseg++;
-			segs[curseg].ds_addr = curaddr;
-			segs[curseg].ds_len = PAGE_SIZE;
+			++*curseg;
+			segs[*curseg].ds_addr = curaddr;
+			segs[*curseg].ds_len = PAGE_SIZE;
 		}
 		lastaddr = curaddr;
 	}
+	return (0);
+}

-	*rsegs = curseg + 1;
+int
+_bus_dmamem_alloc_range(bus_dma_tag_t t, bus_size_t size,
+    bus_size_t alignment, bus_size_t boundary, bus_dma_segment_t *segs,
+    int nsegs, int *rsegs, int flags, bus_addr_t low, bus_addr_t high)
+{
+	int curseg, error;
+	bus_size_t a_size, c_size;
+
+	/* Always round the size. */
+	size = round_page(size);
+
+	if (boundary == 0 || size <= boundary) {
+		curseg = 0;
+		error = __alloc_next_chunk(size, alignment, boundary,
+		  segs, nsegs, &curseg, flags, low, high);
+		if (error)
+			return error;
+		curseg++;
+	} else {
+/*
+ * uvm subsystem will fail to allocate this at once ...
+ * so we need to allocate in smaller peaces.
+ * remark: we need to start a new segment for each chunk!
+ *         otherwise we may violate the boundary conditions.
+ */
+		for (curseg = 0, c_size = 0; c_size < size;
+		    curseg++, c_size += a_size) {
+			if ((a_size = size - c_size) > boundary)
+				a_size = boundary; /* clamp to boundary ... */
+			error = __alloc_next_chunk(a_size, alignment, boundary,
+			  segs, nsegs - curseg, &curseg, flags, low, high);
+			if (error) { /* need to free allocated stuff again ... */
+				if (curseg != 0)
+					_bus_dmamem_free(t, segs, curseg);
+				return error;
+			}
+		}
+	}

+	*rsegs = curseg;
 	return (0);
 }
 #endif /* _BUS_DMAMEM_ALLOC_RANGE */

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.