Now that I learned about possible spurious wakeups this
place needs fixing too. Replace the self-coded sleep variant
with the generic wait_event() helper.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
put_device_state_wait() doesn't loop on the condition and a spurious
wakeup will have it free the device state even though there might still
be references out to it.
Fix this by using 'normal' wait primitives.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
ARM allmodconfig gained a new warning when dma_addr_t is 32-bit wide:
drivers/iommu/arm-smmu.c: In function 'arm_smmu_iova_to_phys_hard':
drivers/iommu/arm-smmu.c:1255:3: warning: right shift count >= width of type
This changes the calculation so that the effective type is always
64-bit.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Fixes: 859a732e4f ("iommu/arm-smmu: add support for iova_to_phys through ATS1PR")
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Currently a PAMU driver patch is very likely to receive some
checkpatch complaints about the code in the context of the
patch. This patch is an attempt to fix most of that and make
the driver more readable
Also fixed a subset of the sparse and coccinelle reported
issues.
Signed-off-by: Emil Medve <Emilian.Medve@Freescale.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Fixes this compile warning:
drivers/iommu/omap-iommu.c: In function 'omap_iommu_map':
drivers/iommu/omap-iommu.c:1139:2: warning: format '%x' expects argument of type 'unsigned int', but argument 5 has type 'phys_addr_t' [-Wformat=]
dev_dbg(dev, "mapping da 0x%lx to pa 0x%x size 0x%x\n", da, pa, bytes);
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
When adding a new device the driver loops over all registered IOMMUs and
calls the ipmmu_find_utlbs() function to parse the DT iommus attribute.
The function returns an error when the IOMMU referenced in DT doesn't
match the current IOMMU. The caller incorrectly breaks from the loop
immediately when the error is reported, resulting in only the first
IOMMU being considered.
Fix this, and while at it move code that isn't specific to an IOMMU
instance out of the loop.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
A lot of the IOMMU support code does not build if the CPU does
not have an MMU itself, and it's not clear if there is any
use case for it, so let's just disable it and wait for anybody
to need it.
This avoids randconfig errors like
../arch/arm/mm/dma-mapping.c: In function '__iommu_alloc_remap':
../arch/arm/mm/dma-mapping.c:1278:34: error: 'VM_ARM_DMA_CONSISTENT' undeclared (first use in this function)
area = get_vm_area_caller(size, VM_ARM_DMA_CONSISTENT | VM_USERMAP,
^
../arch/arm/mm/dma-mapping.c:1278:34: note: each undeclared identifier is reported only once for each function it appears in
../arch/arm/mm/dma-mapping.c: In function '__atomic_get_pages':
../arch/arm/mm/dma-mapping.c:1358:27: error: 'atomic_pool' undeclared (first use in this function)
struct dma_pool *pool = &atomic_pool;
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Section mismatch in reference from the variable fsl_of_pamu_driver to the function .init.text:fsl_pamu_probe()
The variable fsl_of_pamu_driver references
the function __init fsl_pamu_probe()
If the reference is valid then annotate the
variable with __init* or __refdata (see linux/init.h) or name the variable:
*_template, *_timer, *_sht, *_ops, *_probe, *_probe_one, *_console
Signed-off-by: Emil Medve <Emilian.Medve@Freescale.com>
Acked-by: Varun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Replace the custom page table allocation implementation with the
standard allocator.
The driver loses the ability to map 64kB chunkgs using the PTE
contiguous hint, hence the removal of the SZ_64K page size from the
IOMMU page sizes bitmap.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Commit 315786ebbf ("iommu: Add iommu_map_sg() function") adds a new
->map_sg() callback and provides a default implementation that drivers
can use until they implement a hardware-specific variant. Unfortunately
the Tegra GART driver was not updated as part of that commit, so that
iommu_map_sg() calls on a domain provided by the GART cause an oops.
Fixes: 315786ebbf ("iommu: Add iommu_map_sg() function")
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
The driver currently doesn't work as expected and causes existing setups
with Tegra20 to break after commit df06b759f2 ("drm/tegra: Add IOMMU
support"). To restore these setups, do not register the operations with
the platform bus for now. Fixing this properly will involve non-trivial
changes to the DRM driver, which are unlikely to be accepted at this
point in the release cycle.
Reported-by: Misha Komarovskiy <zombah@gmail.com>
Reported-by: Nicolas Chauvet <kwizart@gmail.com>
Tested-by: Misha Komarovskiy <zombah@gmail.com>
Tested-by: Dmitry Osipenko <digetx@gmail.com>
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
iommu_map() calls trace_map() with iova and size. trace_map()
should report original iova and original size as opposed to
iova and size after they get changed during mapping. size is
always zero at the end of mapping which is useless to report
and iova as it gets incremented, it is not as useful as the
original iova. Change iommu_map() to call trace_map() to
report original iova and original size.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Reported-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Commit 7fa1c842ca "iommu/irq_remapping: Change variable
disable_irq_remap to be static" returns unconditionally success from
the irq remapping prepare callback if the iommu can be initialized.
The change assumed that iommu_go_to_state(IOMMU_ACPI_FINISHED) returns
a failure if irq remapping is not enabled, but thats not the case.
The function returns success when the iommu is initialized to the
point which is required for remapping to work. The actual state of the
irq remapping feature is reflected in the status variable
amd_iommu_irq_remap, which is not considered in the return value.
The fix is simple: If the iommu_go_to_state() returns success,
evaluate the remapping state amd_iommu_irq_remap and reflect it in the
return value.
Fixes: 7fa1c842ca iommu/irq_remapping: Change variable disable_irq_remap to be static
Reported-and-tested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Joerg Roedel <joro@8bytes.org>
Currently, we provide the iommu_ops.iova_to_phys service by doing a
table walk in software to translate IO virtual addresses to physical
addresses. On SMMUs that support it, it can be useful to ask the SMMU
itself to do the translation. This can be used to warm the TLBs for an
SMMU. It can also be useful for testing and hardware validation.
Since the address translation registers are optional on SMMUv2, only
enable hardware translations when using SMMUv1 or when SMMU_IDR0.S1TS=1
and SMMU_IDR0.ATOSNS=0, as described in the ARM SMMU v1-v2 spec.
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
[will: reworked on top of generic iopgtbl changes]
Signed-off-by: Will Deacon <will.deacon@arm.com>
Currently we do a STLBIALL when we initialize the SMMU. However, in
some configurations that register is not supposed to be touched and is
marked as "Secure only" in the spec. Rip it out.
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
This patch adds a series of basic self-consistency tests to the ARM LPAE
IO page table allocator that exercise corner cases in map/unmap, as well
as testing all valid configurations of pagesize, ias and stage.
Signed-off-by: Will Deacon <will.deacon@arm.com>
A number of IOMMUs found in ARM SoCs can walk architecture-compatible
page tables.
This patch adds a generic allocator for Stage-1 and Stage-2 v7/v8
long-descriptor page tables. 4k, 16k and 64k pages are supported, with
up to 4-levels of walk to cover a 48-bit address space.
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
This patch introduces a generic framework for allocating page tables for
an IOMMU. There are a number of reasons we want to do this:
- It avoids duplication of complex table management code in IOMMU
drivers that use the same page table format
- It removes any coupling with the CPU table format (and even the
architecture!)
- It defines an API for IOMMU TLB maintenance
Tested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Currently map and unmap are implemented as events under a
common trace class declaration. The common class forces
trace_unmap() to require a bogus physical address argument
that it doesn't use. Changing unmap to report unmapped size
will provide useful information for debugging. Remove common
map_unmap trace class and change map and unmap into separate
events as opposed to events under the same class to allow for
differences in the reporting information. In addition, map and
unmap are changed to handle size value as size_t instead of int
to match the passed size value and avoid overflow.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Suggested-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
iommu_unmap() calls trace_unmap() with changed iova and original
size. trace_unmap() should report original iova instead. Change
iommu_unmap() to call trace_unmap() with original iova.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Reported-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Systems may contain heterogeneous IOMMUs supporting differing minimum
page sizes, which may also not be common with the CPU page size.
Thus it is practical to have an explicit notion of IOVA granularity
to simplify handling of mapping and allocation constraints.
As an initial step, move the IOVA page granularity from an implicit
compile-time constant to a per-domain property so we can make use
of it in IOVA domain context at runtime. To keep the abstraction tidy,
extend the little API of inline iova_* helpers to parallel some of the
equivalent PAGE_* macros.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
To share the IOVA allocator with other architectures, it needs to
accommodate more general aperture restrictions; move the lower limit
from a compile-time constant to a runtime domain property to allow
IOVA domains with different requirements to co-exist.
Also reword the slightly unclear description of alloc_iova since we're
touching it anyway.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
In order to share the IOVA allocator with other architectures, break
the unnecssary dependency on the Intel IOMMU driver and move the
remaining IOVA internals to iova.c
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
In preparation for sharing the IOVA allocator, split it out under its
own Kconfig symbol.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
No board file instantiates the IPMMU using platform data. Now that we
have DT support, get rid of platform data.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Devices such as the system DMA controller are connected to multiple
micro TLBs of the same IOMMU. Support this.
Selective enabling of micro TLBs based on runtime device usage isn't
possible at the moment due to lack of support in the IOMMU and DMA
mapping APIs. Support for devices connected to different IOMMUs is also
unsupported for the same reason.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Make platform data optional when the device is instantiated from DT and
look up the micro-TLB number in the bus master DT node.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
The TLB must be invalidated after unmapping memory to remove stale TLB
entries. this was supposed to be performed already, but a bug in the
driver prevented the TLB invalidate function from being called. Fix it.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
When clearing PUD or PMD entries the child page table (if any) is freed
and the PUD or PMD entry is then cleared. This result in a small race
condition window during which a free page table could be accessed by the
IPMMU.
Fix it by clearing and flushing the PUD or PMD entry before freeing the
child page table.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
The ARM IOMMU mapping needs to be released when attaching the device
fails. Add arm_iommu_release_mapping() to the error code path. This is
safe to call with a NULL mapping, so no specific check is needed.
Cleanup is also missing when failing to create a mapping. Jump to the
error code path in that case instead of returning immediately.
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Currently if CPU supports X2APIC, IR hardware must work in X2APIC mode
or disabled. Change the code to support IR working in XAPIC mode when
CPU supports X2APIC. Then the CPU APIC driver will decide how to handle
such as configuration by:
1) Disabling X2APIC mode
2) Forcing X2APIC physical mode
This change also fixes a live locking when
1) BIOS enables CPU X2APIC
2) DMAR table disables X2APIC mode or IR hardware doesn't support X2APIC
with following messages:
[ 37.863463] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2
[ 37.863463] INTR-REMAP:[fault reason 36] Detected reserved fields in the IRTE entry
[ 37.879372] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2
[ 37.879372] INTR-REMAP:[fault reason 36] Detected reserved fields in the IRTE entry
[ 37.895282] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2
[ 37.895282] INTR-REMAP:[fault reason 36] Detected reserved fields in the IRTE entry
[ 37.911192] dmar: INTR-REMAP: Request device [[f0:1f.7] fault index 2
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Tested-by: Joerg Roedel <joro@8bytes.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: iommu@lists.linux-foundation.org
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Link: http://lkml.kernel.org/r/1420615903-28253-11-git-send-email-jiang.liu@linux.intel.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The whole iommu setup for irq remapping is a convoluted mess. The
iommu detect function gets called from mem_init() and the prepare
callback gets called from enable_IR_x2apic() for unknown reasons.
Of course AMD and Intel setup differs in nonsensical ways. Intels
prepare callback is explicit while AMDs prepare callback is implicit
in setup_irq_remapping_ops() just to be called in the prepare call
again.
Because all of this gets called from enable_IR_x2apic() and the dmar
prepare function merily parses the ACPI tables, but does not allocate
memory we end up with memory allocation from irq disabled context
later on.
AMDs iommu code at least allocates the required memory from the
prepare function. That has issues as well, but thats not scope of this
patch.
The goal of this change is to distangle the allocation from the actual
enablement. There is no point to allocate memory from irq disabled
regions with GFP_ATOMIC just because it does not matter at that point
in the boot stage. It matters with physical hotplug later on.
There is another issue with the current setup. Due to the conversion
to stacked irqdomains we end up with a call into the irqdomain
allocation code from irq disabled context, but that code does
GFP_KERNEL allocations rightfully as there is no reason to do
preperatory allocations with GFP_ATOMIC.
That change caused the allocator code to complain about GFP_KERNEL
allocations invoked in atomic context. Boris provided a temporary
hackaround which changed the GFP flags if irq_domain_add() got called
from atomic context. Not pretty and we really dont want to get this
into a mainline release for obvious reasons.
Move the ACPI table parsing and the resulting memory allocations from
the enable to the prepare function. That allows to get rid of the
horrible hackaround in irq_domain_add() later.
[Jiang] Rebased onto v3.19
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Borislav Petkov <bp@alien8.de>
Acked-and-tested-by: Joerg Roedel <joro@8bytes.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: iommu@lists.linux-foundation.org
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: x86@kernel.org
Link: http://lkml.kernel.org/r/20141205084147.313026156@linutronix.de
Link: http://lkml.kernel.org/r/1420615903-28253-3-git-send-email-jiang.liu@linux.intel.com
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
This platform_driver does not need to set an owner, it will be populated by the
driver core.
Signed-off-by: Wolfram Sang <wsa@the-dreams.de>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
