Re: WARNING: at arch/arm/mm/consistent.c:368 dma_free_coherent | |
| [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] | |
Mikael Pettersson writes:
> Russell King - ARM Linux writes:
> > On Sun, Aug 24, 2008 at 09:17:56PM +0200, Mikael Pettersson wrote:
> > > >On Mon, Aug 18, 2008 at 08:31:26AM +0100, Russell King - ARM Linux wrote:
> > > >> On Sun, Aug 17, 2008 at 07:39:51PM +0200, Mikael Pettersson wrote:
> > > >> > FWIW, my ixp4xx ds101 machine has been seeing these dma_free_coherent
> > > >> > warnings for ever, with both the IDE (aec62xx) and libata (pata_artop)
> > > >> > drivers for its IDE controller. The machine is currently running 2.6.27-rc3
> > > >> > plus this consistent gc patch, and so far things are looking good.
> > > >>
> > > >> Thanks.
> > > >>
> > > >> However, having looked at the original report, I think a better
> > > >> solution would be to try and stop IXP4xx from using the dmabounce
> > > >> code.
> > > >>
> > > >> Can you try removing my patch, and changing dma_set_mask() in
> > > >> arch/arm/include/asm/dma-mapping.h so it doesn't set the dma mask
> > > >> to have more bits set than the current mask?
> > > >
> > > >And do the same thing to pci_set_dma_mask and pci_set_consistent_dma_mask
> > > >in drivers/pci/pci.c
> > >
> > > Ok, I did the following:
> > > - changed dma-mapping.h to first WARN_ON if the given mask exceeds the
> > > current mask, and then update the mask with &= not just =
> > > - ixp4xx overrides pci_set_{,consistent_}dma_mask (the ixp4xx hardware.h
> > > sets HAVE_ARCH_PCI_SET_DMA_MASK), so I changed those routines in
> > > mach-ixp4xx/pci-common.c not drivers/pci/pci.c
> > > - updated mach-ixp4xx/Kconfig and mach-ixp4xx/pci-common.c so that
> > > dmabounce.c could be eliminated
> > >
> > > (patch at the end of this email)
> > >
> > > This didn't work very well. There are two WARN_ONs when libata
> > > tries to assign ATA_DMA_MASK (2^32 - 1) to the PCI controller.
> > > The disk is detected ok (but detection is done with pio), but when
> > > init starts accessing the root file system the machine just hangs
> > > without any further kernel messages:
> >
> > I wonder if we're still getting invalid DMA addresses through. You
> > can check that by leaving the DMA bounce code in place, but making
> > it BUG_ON() or something when a buffer with a wrong DMA mask is
> > passed.
>
> I didn't have time to try this.
>
> > The other place to look is drivers/scsi/scsi_lib.c - check whether
> > scsi_calculate_bounce_limit() is returning the right thing (iow,
> > the proper mask value.) That should be passed to
> > blk_queue_bounce_limit() to make the block layer do the bouncing.
>
> But I did try this. It turns out that the logic for enabling bounce
> buffering in both the scsi layer and old IDE depends on the
> platform defining PCI_DMA_BUS_IS_PHYS with a non-zero value.
> But arch/arm/include/asm/pci.h defines it as the constant zero,
> so scsi_calculate_bounce_limit() returns BLK_BOUNCE_ANY ("don't
> bounce anything") which is passed on to blk_queue_bounce_limit().
>
> Another breakage is drivers/ata/libata-sff.c unconditionally passing
> ATA_DMA_MASK (-1U) to pci_set_{,consistent_}dma_mask().
>
> However, even with these two fixed (set PCI_DMA_BUS_IS_PHYS to 1
> and ATA_DMA_MASK to 0x03ffffff) the arm dmabounce code is needed:
> if I also disable the dmabounce code my ixp4xx hangs when init tries
> to remount / read-write.
>
> I'll try to do some more debugging next weekend.
I've now traced the actions in dmabounce.c, and I _think_ I
understand what's going on.
First, my ds101 ixp420 box has 64MB of RAM, so it shouldn't need to
ever bounce any buffers. But based on my previous experiments it
clearly needed some bounce support because otherwise the machine
hangs when init remounts / read-write.
Tracing dmabounce.c:map_single() showed that bouncing never triggered
due to the dma_mask, because clearly no page ever resided above it.
However, mach-ixp4xx/common-pci.c:dma_needs_bounce() returned true
whenever the very last page at 64MB-4096 was accessed. dma_needs_bounce()
tests dma_addr + size >= SZ_64M, but that's actually off-by-one.
Changing it to dma_addr + size - 1 >= SZ_64M disabled the bounces,
but also caused the system hang to reappear.
Clearly that last page is somehow "special".
I then changed mach-ixp4xx/common-pci.c to set the dma masks as
64MB - 4096 - 1 rather than 64MB - 1, so as to exclude the last page.
That triggered errors in dmabounce.c and consistent.c because code
there assumes that dma masks are (2^N)-1 which breaks in this case.
With those places fixed (I think...) and scsi_calculate_bounce_limit()
hacked to ignore the bogus PCI_DMA_BUS_IS_PHYS, my kernel now boots
fine and works under file system stress without triggering bounces
from arm's dmabounce.c, the block and mm layers do the bouncing for us.
After reading the IXP4xx specification update, I believe the reason
the last page doesn't work for PCI DMA is IXP4xx erratum 15 (SCR 1289).
The PCI controller prefetches data when an external PCI master reads
memory. These prefetches can cross the boundary between valid memory
and a reserved region, causing an error on the AHB bus. This error is
not always cleared, leading to infinite retries and a PCI lock-up.
/Mikael
--- linux-2.6.27-rc5/arch/arm/common/dmabounce.c.~1~ 2008-08-29 13:08:50.000000000 +0200
+++ linux-2.6.27-rc5/arch/arm/common/dmabounce.c 2008-09-07 21:11:43.000000000 +0200
@@ -212,18 +212,21 @@ map_single(struct device *dev, void *ptr
struct dmabounce_device_info *device_info = dev->archdata.dmabounce;
dma_addr_t dma_addr;
int needs_bounce = 0;
+ unsigned long mask;
if (device_info)
DO_STATS ( device_info->map_op_count++ );
dma_addr = virt_to_dma(dev, ptr);
+ mask = 0;
if (dev->dma_mask) {
- unsigned long mask = *dev->dma_mask;
unsigned long limit;
- limit = (mask + 1) & ~mask;
- if (limit && size > limit) {
+ mask = *dev->dma_mask;
+
+ limit = mask + 1;
+ if (limit && size >= limit) {
dev_err(dev, "DMA mapping too big (requested %#x "
"mask %#Lx)\n", size, *dev->dma_mask);
return ~0;
@@ -232,10 +235,19 @@ map_single(struct device *dev, void *ptr
/*
* Figure out if we need to bounce from the DMA mask.
*/
- needs_bounce = (dma_addr | (dma_addr + size - 1)) & ~mask;
+ needs_bounce = dma_addr > mask || dma_addr + size - 1 > mask;
+ if (needs_bounce)
+ printk("%s: dev %p mask %#lx dma_addr %#x size %#x sets needs_bounce\n",
+ __FUNCTION__, dev, mask, dma_addr, size);
+ }
+
+ if (device_info && !needs_bounce && dma_needs_bounce(dev, dma_addr, size)) {
+ needs_bounce = 1;
+ printk("%s: dma_needs_bounce(%p, %#x, %#x) mask %#lx sets needs_bounce\n",
+ __FUNCTION__, dev, dma_addr, size, mask);
}
- if (device_info && (needs_bounce || dma_needs_bounce(dev, dma_addr, size))) {
+ if (device_info && needs_bounce/*(needs_bounce || dma_needs_bounce(dev, dma_addr, size))*/) {
struct safe_buffer *buf;
buf = alloc_safe_buffer(device_info, ptr, size, dir);
@@ -258,6 +270,8 @@ map_single(struct device *dev, void *ptr
}
ptr = buf->safe;
+ printk("%s: dev %p mask %#lx old_dma_addr %#x new_dma_addr %#x\n",
+ __FUNCTION__, dev, mask, dma_addr, buf->safe_dma_addr);
dma_addr = buf->safe_dma_addr;
} else {
/*
--- linux-2.6.27-rc5/arch/arm/mach-ixp4xx/common-pci.c.~1~ 2008-08-29 13:08:51.000000000 +0200
+++ linux-2.6.27-rc5/arch/arm/mach-ixp4xx/common-pci.c 2008-09-07 20:09:01.000000000 +0200
@@ -321,8 +321,8 @@ static int abort_handler(unsigned long a
static int ixp4xx_pci_platform_notify(struct device *dev)
{
if(dev->bus == &pci_bus_type) {
- *dev->dma_mask = SZ_64M - 1;
- dev->coherent_dma_mask = SZ_64M - 1;
+ *dev->dma_mask = SZ_64M - 4096 - 1;
+ dev->coherent_dma_mask = SZ_64M - 4096 - 1;
dmabounce_register_dev(dev, 2048, 4096);
}
return 0;
--- linux-2.6.27-rc5/arch/arm/mach-ixp4xx/include/mach/memory.h.~1~ 2008-08-29 13:08:51.000000000 +0200
+++ linux-2.6.27-rc5/arch/arm/mach-ixp4xx/include/mach/memory.h 2008-09-07 20:15:58.000000000 +0200
@@ -21,7 +21,7 @@ void ixp4xx_adjust_zones(int node, unsig
#define arch_adjust_zones(node, size, holes) \
ixp4xx_adjust_zones(node, size, holes)
-#define ISA_DMA_THRESHOLD (SZ_64M - 1)
+#define ISA_DMA_THRESHOLD (SZ_64M - 4096 - 1)
#endif
--- linux-2.6.27-rc5/arch/arm/mm/consistent.c.~1~ 2008-09-07 17:22:54.000000000 +0200
+++ linux-2.6.27-rc5/arch/arm/mm/consistent.c 2008-09-07 21:09:43.000000000 +0200
@@ -183,7 +183,7 @@ __dma_alloc(struct device *dev, size_t s
* Sanity check the allocation size.
*/
size = PAGE_ALIGN(size);
- limit = (mask + 1) & ~mask;
+ limit = mask + 1;
if ((limit && size >= limit) ||
size >= (CONSISTENT_END - CONSISTENT_BASE)) {
printk(KERN_WARNING "coherent allocation too big "
--- linux-2.6.27-rc5/block/blk-settings.c.~1~ 2008-08-29 13:08:52.000000000 +0200
+++ linux-2.6.27-rc5/block/blk-settings.c 2008-09-07 19:59:05.000000000 +0200
@@ -134,6 +134,8 @@ void blk_queue_bounce_limit(struct reque
unsigned long b_pfn = dma_addr >> PAGE_SHIFT;
int dma = 0;
+ printk("%s: q %p dma_addr %#llx max_low_pfn %#lx blk_max_low_pfn %#lx\n",
+ __FUNCTION__, q, dma_addr, max_low_pfn, blk_max_low_pfn);
q->bounce_gfp = GFP_NOIO;
#if BITS_PER_LONG == 64
/* Assume anything <= 4GB can be handled by IOMMU.
--- linux-2.6.27-rc5/drivers/scsi/scsi_lib.c.~1~ 2008-08-29 13:08:53.000000000 +0200
+++ linux-2.6.27-rc5/drivers/scsi/scsi_lib.c 2008-09-07 19:55:52.000000000 +0200
@@ -1615,13 +1615,15 @@ u64 scsi_calculate_bounce_limit(struct S
* Platforms with virtual-DMA translation
* hardware have no practical limit.
*/
- if (!PCI_DMA_BUS_IS_PHYS)
+ if (0 && !PCI_DMA_BUS_IS_PHYS)
return BLK_BOUNCE_ANY;
host_dev = scsi_get_device(shost);
if (host_dev && host_dev->dma_mask)
bounce_limit = *host_dev->dma_mask;
+ printk("%s: returning bounce_limit %#llx for dev %p\n",
+ __FUNCTION__, bounce_limit, host_dev);
return bounce_limit;
}
EXPORT_SYMBOL(scsi_calculate_bounce_limit);
-------------------------------------------------------------------
List admin: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ: http://www.arm.linux.org.uk/mailinglists/faq.php
Etiquette: http://www.arm.linux.org.uk/mailinglists/etiquette.php
[Site Home] [Linux Arm] [Fedora ARM] [Gcc Help] [Git] [DCCP] [IETF Announce] [Security] [PDAs] [Linux] [Linux Book List] [Linux MIPS] [Yosemite Campsites] [Photos]
|
|