Commit Graph

170 Commits

Author SHA1 Message Date
Christopher Snowhill
1799c761c4 drm/xe: Validate uAPI padding and reserved fields
Padding and reserved fields are declared such that they must be
zeroed, so verify that they're all zero in the respective ioctl
functions.

Derived from original patch by mlankhorst.

v2:
	Removed extensions checks where there were none originally. (José)
	Moved extraneous parentheses to the correct places. (Lucas)

Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Christopher Snowhill <kode54@gmail.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:34:01 -05:00
Niranjana Vishwanathapura
1b1d371038 drm/xe: Apply upper limit to sg element size
The iommu_dma_map_sg() function ensures iova allocation doesn't
cross dma segment boundary. It does so by padding some sg elements.
This can cause overflow, ending up with sg->length being set to 0.
Avoid this by halving the maximum segment size (rounded down to
PAGE_SIZE).

Specify maximum segment size for sg elements by using
sg_alloc_table_from_pages_segment() to allocate sg_table.

v2: Use correct max segment size in dma_set_max_seg_size() call

Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Bruce Chang <yu.bruce.chang@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:34:01 -05:00
Rodrigo Vivi
328f3414b1 drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.
There are multiple kind of config prints and with the upcoming
devcoredump there will be another layer. Let's limit the config
to the top level functions and leave the clean-up work for the
compilers so we don't create a spider-web of configs.

No functional change. Just a preparation for the devcoredump.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
2023-12-19 18:33:53 -05:00
Matthew Brost
ed1df98974 drm/xe: Allow compute VMs to output dma-fences on binds
Binds are not long running jobs thus we can export dma-fences even if a
VM is in compute mode.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:33:51 -05:00
Niranjana Vishwanathapura
9ca14f94d2 drm/xe: Handle -EDEADLK case in preempt worker
With multiple active VMs, under memory pressure, it is possible that
ttm_bo_validate() run into -EDEADLK in ttm_mem_evict_wait_busy() and
return -ENOMEM.

Until ttm properly handles locking in such scenarios, best thing the
driver can do is unwind the lock and retry.

Update preempt worker to retry validating BOs with a timeout upon
-ENOMEM.

v2: revert retry timeout upon -EAGAIN (Thomas)

Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:33:49 -05:00
Francois Dugast
116d325152 drm/xe: Fix splat during error dump
Allow xe_bo_addr without lock to print debug information, such
as from xe_analyze_vm.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:33:13 -05:00
Lucas De Marchi
58e19acf0c drm/xe: Cleanup page-related defines
Rename the following defines to lose the GEN* prefixes since they don't
make sense for xe:

GEN8_PTE_SHIFT		-> XE_PTE_SHIFT
GEN8_PAGE_SIZE		-> XE_PAGE_SIZE
GEN8_PTE_MASK		-> XE_PTE_MASK
GEN8_PDE_SHIFT		-> XE_PDE_SHIFT
GEN8_PDES		-> XE_PDES
GEN8_PDE_MASK		-> XE_PDE_MASK
GEN8_64K_PTE_SHIFT	-> XE_64K_PTE_SHIFT
GEN8_64K_PAGE_SIZE	-> XE_64K_PAGE_SIZE
GEN8_64K_PTE_MASK	-> XE_64K_PTE_MASK
GEN8_64K_PDE_MASK	-> XE_64K_PDE_MASK
GEN8_PDE_PS_2M		-> XE_PDE_PS_2M
GEN8_PDPE_PS_1G		-> XE_PDPE_PS_1G
GEN8_PDE_IPS_64K	-> XE_PDE_IPS_64K
GEN12_GGTT_PTE_LM	-> XE_GGTT_PTE_LM
GEN12_USM_PPGTT_PTE_AE	-> XE_USM_PPGTT_PTE_AE
GEN12_PPGTT_PTE_LM	-> XE_PPGTT_PTE_LM
GEN12_PDE_64K		-> XE_PDE_64K
GEN12_PTE_PS64		-> XE_PTE_PS64
GEN8_PAGE_PRESENT	-> XE_PAGE_PRESENT
GEN8_PAGE_RW		-> XE_PAGE_RW
PTE_READ_ONLY		-> XE_PTE_READ_ONLY

Keep an XE_ prefix to make sure we don't mix the defines for the CPU
(e.g. PAGE_SIZE) with the ones fro the GPU).

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:31:43 -05:00
Matthew Brost
cf667aec0a drm/xe: Decrement fault mode counts in xe_vm_close_and_put
Rather waiting for the VM to be destroyed (all refs to VM go to zero),
drop the fault mode counts when the VM is closed in xe_vm_close_and_put.
This avoids a window where user space can create a faulting VM, close
it, and a subsequent creation of a non-faulting VM fails.

v2 (Lucas): Drop VLK reference in commit message

Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Suggested-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:30:22 -05:00
Thomas Hellström
8e41443e1b drm/xe/vm: Defer vm rebind until next exec if nothing to execute
If all compute engines of a vm in compute mode are idle,
defer a rebind to the next exec to avoid the VM unnecessarily trying
to make memory resident and compete with other VMs for available
memory space.

Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:30:14 -05:00
Thomas Hellström
4b1430f775 drm/xe/vm: Use the correct vma destroy sequence on userptr failure
Fix the below warning by using the correct vma destroy sequence:

[   92.204921] ------------[ cut here ]------------
[   92.204954] WARNING: CPU: 3 PID: 2449 at drivers/gpu/drm/xe/xe_vm.c:933 xe_vma_destroy+0x280/0x290 [xe]
[   92.205002] Modules linked in: ccm nft_objref cmac nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw iptable_security ip_set nf_tables nfnetlink ip6table_filter iptable_filter bnep sunrpc vfat fat iwlmvm mac80211 intel_rapl_msr ee1004 ppdev intel_rapl_common snd_hda_codec_realtek libarc4 iTCO_wdt snd_hda_codec_generic intel_pmc_bxt x86_pkg_temp_thermal iTCO_vendor_support intel_powerclamp coretemp intel_cstate iwlwifi btusb btrtl btbcm snd_hda_intel btintel snd_intel_dspcfg eeepc_wmi snd_hda_codec asus_wmi bluetooth snd_hwdep snd_seq ledtrig_audio snd_hda_core snd_seq_device sparse_keymap cfg80211 snd_pcm intel_uncore joydev platform_profile mei_me wmi_bmof intel_wmi_thunderbolt snd_timer pcspkr ecdh_generic i2c_i801 snd
[   92.205060]  ecc mei rfkill soundcore idma64 i2c_smbus parport_pc parport acpi_pad acpi_tad xe drm_ttm_helper ttm i2c_algo_bit drm_suballoc_helper kunit drm_buddy gpu_sched drm_display_helper drm_kms_helper drm crct10dif_pclmul crc32_pclmul crc32c_intel nvme nvme_core e1000e ghash_clmulni_intel drm_panel_orientation_quirks video wmi pinctrl_tigerlake usb_storage ip6_tables ip_tables fuse
[   92.205242] CPU: 3 PID: 2449 Comm: xe_vm Tainted: G     U             6.1.0+ #120
[   92.205254] Hardware name: ASUS System Product Name/PRIME B560M-A AC, BIOS 0403 01/26/2021
[   92.205266] RIP: 0010:xe_vma_destroy+0x280/0x290 [xe]
[   92.205299] Code: 74 15 48 8b 93 a0 01 00 00 48 8b 83 a8 01 00 00 48 89 42 08 48 89 10 4c 89 ab a0 01 00 00 4c 89 ab a8 01 00 00 e9 1b fe ff ff <0f> 0b e9 a3 fe ff ff 0f 0b e9 82 fe ff ff 66 90 0f 1f 44 00 00 48
[   92.205322] RSP: 0018:ffffaadd465c3a58 EFLAGS: 00010246
[   92.205331] RAX: 0000000000000000 RBX: ffff9706d53ed400 RCX: 0000000000000001
[   92.205341] RDX: ffff9706d53ed480 RSI: ffffffffa756dc2b RDI: ffffffffa760a05e
[   92.205351] RBP: 0000000000000000 R08: 0000000000000000 R09: 000000002c5370a2
[   92.205361] R10: ffff9706ca520000 R11: 0000000022c5370a R12: ffff9706cad03800
[   92.205370] R13: 000000000004ffff R14: fffffffffffffff2 R15: 0000000000000000
[   92.205380] FS:  00007fe98203a940(0000) GS:ffff970dffac0000(0000) knlGS:0000000000000000
[   92.205392] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   92.205400] CR2: 00007fe982ccb000 CR3: 000000010d6e6003 CR4: 0000000000770ee0
[   92.205410] PKRU: 55555554
[   92.205415] Call Trace:
[   92.205419]  <TASK>
[   92.205426]  vm_bind_ioctl_lookup_vma+0x9bb/0xbf0 [xe]
[   92.205461]  ? lock_is_held_type+0xe3/0x140
[   92.205472]  ? xe_vm_find_overlapping_vma+0x77/0x90 [xe]
[   92.205503]  ? __vm_bind_ioctl_lookup_vma.constprop.0+0x9e/0xe0 [xe]
[   92.205533]  ? __lock_acquire+0x3a3/0x1fb0
[   92.205543]  ? register_lock_class+0x38/0x480
[   92.205550]  ? __lock_acquire+0x3a3/0x1fb0
[   92.205558]  ? __lock_acquire+0x3a3/0x1fb0
[   92.205567]  ? __lock_acquire+0x3a3/0x1fb0
[   92.205579]  ? lock_acquire+0xbf/0x2b0
[   92.205586]  ? lock_acquire+0xcf/0x2b0
[   92.205597]  xe_vm_bind_ioctl+0x977/0x1c30 [xe]
[   92.205630]  ? find_held_lock+0x2b/0x80
[   92.205640]  ? lock_release+0x131/0x2c0
[   92.205648]  ? xe_vm_ttm_bo+0x40/0x40 [xe]
[   92.205677]  drm_ioctl_kernel+0xa1/0x150 [drm]
[   92.205706]  drm_ioctl+0x221/0x420 [drm]
[   92.205727]  ? xe_vm_ttm_bo+0x40/0x40 [xe]
[   92.205764]  __x64_sys_ioctl+0x8d/0xd0
[   92.205774]  do_syscall_64+0x37/0x90
[   92.205781]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[   92.205790] RIP: 0033:0x7fe982be8d6f
[   92.205797] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 18 48 8b 44 24 18 64 48 2b 04 25 28 00 00
[   92.205821] RSP: 002b:00007ffde9f9c560 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[   92.205832] RAX: ffffffffffffffda RBX: 00007fadeadbe000 RCX: 00007fe982be8d6f
[   92.205842] RDX: 00007ffde9f9c5f0 RSI: 0000000040786445 RDI: 0000000000000003
[   92.205851] RBP: 00007ffde9f9c5f0 R08: 00007fadeadbe000 R09: 0000000000040000
[   92.205861] R10: 0000000000000003 R11: 0000000000000246 R12: 0000000040786445
[   92.205871] R13: 0000000000000003 R14: 0000000000000003 R15: 00007fe982e02000
[   92.205888]  </TASK>
[   92.205892] irq event stamp: 82723
[   92.205897] hardirqs last  enabled at (82731): [<ffffffffa617660e>] __up_console_sem+0x5e/0x70
[   92.205910] hardirqs last disabled at (82738): [<ffffffffa61765f3>] __up_console_sem+0x43/0x70
[   92.205922] softirqs last  enabled at (82182): [<ffffffffa60f026d>] __irq_exit_rcu+0xed/0x160
[   92.205935] softirqs last disabled at (82163): [<ffffffffa60f026d>] __irq_exit_rcu+0xed/0x160
[   92.205947] ---[ end trace 0000000000000000 ]---

Reported-by: Francois Dugast <francois.dugast@intel.com>
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:29:45 -05:00
Matthew Auld
2a8477f761 drm/xe: s/lmem/vram/
This seems to be the preferred nomenclature in xe. Currently we are
intermixing vram and lmem, which is confusing.

v2 (Gwan-gyeong Mun & Lucas):
  - Rather apply to the entire driver

Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Acked-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:29:45 -05:00
Lucas De Marchi
ea9f879d03 drm/xe: Sort includes
Sort includes and split them in blocks:

1) .h corresponding to the .c. Example: xe_bb.c should have a "#include
   "xe_bb.h" first.
2) #include <linux/...>
3) #include <drm/...>
4) local includes
5) i915 includes

This is accomplished by running
`clang-format --style=file -i --sort-includes drivers/gpu/drm/xe/*.[ch]`
and ignoring all the changes after the includes. There are also some
manual tweaks to split the blocks.

v2: Also sort includes in headers

Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:29:20 -05:00
Matthew Brost
5669899e9b drm/xe: Add has_asid to device info
Rather than alias supports_usm to ASIS support, add an explicit
variable to indicate ASID support.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
2023-12-19 18:27:47 -05:00
Matthew Brost
da3799c975 drm/xe: Use GuC to do GGTT invalidations for the GuC firmware
Only the GuC should be issuing TLB invalidations if it is enabled. Part
of this patch is sanitize the device on driver unload to ensure we do
not send GuC based TLB invalidations during driver unload.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
2023-12-19 18:27:47 -05:00
Matthew Brost
74a8b2c6e2 drm/xe: Propagate error from bind operations to async fence
If an bind operation fails we need to report it via the async fence.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
2023-12-19 18:27:46 -05:00
Matthew Brost
332dd0116c drm/xe: Add range based TLB invalidations
If the platform supports range based TLB invalidations use them. Hide
these details in the xe_gt_tlb_invalidation layer.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
2023-12-19 18:27:46 -05:00
Matthew Brost
a12d921674 drm/xe: Only set VM->asid for platforms that support a ASID
This will help with TLB invalidation as the ASID in TLB invalidate
should be zero for platforms that do not support a ASID.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
2023-12-19 18:27:46 -05:00
Matthew Brost
fc108a8b75 drm/xe: Add TLB invalidation fence
Fence will be signaled when TLB invalidation completion.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Suggested-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:27:45 -05:00
Matthew Brost
a9351846d9 drm/xe: Break of TLB invalidation into its own file
TLB invalidation is used by more than USM (page faults) so break this
code out into its own file.

Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
2023-12-19 18:27:45 -05:00
Matthew Brost
dd08ebf6c3 drm/xe: Introduce a new DRM driver for Intel GPUs
Xe, is a new driver for Intel GPUs that supports both integrated and
discrete platforms starting with Tiger Lake (first Intel Xe Architecture).

The code is at a stage where it is already functional and has experimental
support for multiple platforms starting from Tiger Lake, with initial
support implemented in Mesa (for Iris and Anv, our OpenGL and Vulkan
drivers), as well as in NEO (for OpenCL and Level0).

The new Xe driver leverages a lot from i915.

As for display, the intent is to share the display code with the i915
driver so that there is maximum reuse there. But it is not added
in this patch.

This initial work is a collaboration of many people and unfortunately
the big squashed patch won't fully honor the proper credits. But let's
get some git quick stats so we can at least try to preserve some of the
credits:

Co-developed-by: Matthew Brost <matthew.brost@intel.com>
Co-developed-by: Matthew Auld <matthew.auld@intel.com>
Co-developed-by: Matt Roper <matthew.d.roper@intel.com>
Co-developed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Co-developed-by: Francois Dugast <francois.dugast@intel.com>
Co-developed-by: Lucas De Marchi <lucas.demarchi@intel.com>
Co-developed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Co-developed-by: Philippe Lecluse <philippe.lecluse@intel.com>
Co-developed-by: Nirmoy Das <nirmoy.das@intel.com>
Co-developed-by: Jani Nikula <jani.nikula@intel.com>
Co-developed-by: José Roberto de Souza <jose.souza@intel.com>
Co-developed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Co-developed-by: Dave Airlie <airlied@redhat.com>
Co-developed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Co-developed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Co-developed-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
2023-12-12 14:05:48 -05:00