Previously, compressible surfaces were required to be non-coherent
(allocated as WC) because compression and coherency were mutually
exclusive. Starting with Xe3, hardware supports combining compression
with 1-way coherency, allowing compressible surfaces to be allocated as
WB memory. This provides applications with more efficient memory
allocation by avoiding WC allocation overhead that can cause system
stuttering and memory management challenges.
The implementation adds support for compressed+coherent PAT entry for
the xe3_lpg devices and updates the driver logic to handle the new
compression capabilities.
v2: (Matthew Auld)
- Improved error handling with XE_IOCTL_DBG()
- Enhanced documentation and comments
- Fixed xe_bo_needs_ccs_pages() outdated compression assumptions
v3:
- Improve WB compression support detection by checking PAT table
instead of version check
v4:
- Add XE_CACHE_WB_COMPRESSION, which simplifies the logic.
v5:
- Use U16_MAX for the invalid PAT index. (Matthew Auld)
Bspec: 71582, 59361, 59399
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Xin Wang <x.wang@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260109093007.546784-1-x.wang@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
There are additional hardware managed L2$ flushing such as the
transient display. In those scenarios, page reclamation is
unnecessary resulting in redundant cacheline flushes, so skip
over those corresponding ranges.
v2:
- Elaborated on reasoning for page reclamation skip based on
Tejas's discussion. (Matthew A, Tejas)
v3:
- Removed MEDIA_IS_ON due to racy condition resulting in removal of
relevant registers and values. (Matthew A)
- Moved l3 policy access to xe_pat. (Matthew A)
v4:
- Updated comments based on previous change. (Tejas)
- Move back PAT index macros to xe_pat.c.
Signed-off-by: Brian Nguyen <brian3.nguyen@intel.com>
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20251212213225.3564537-21-brian3.nguyen@intel.com
The existing "pat" debugfs node dumps the live PAT registers. Under
SR-IOV the VF cannot touch those registers, so the file vanishes and
users lose all PAT visibility. Add a VF-safe "pat_sw_config" entry to
the VF-safe debugfs list. It prints the cached PAT table the driver
programmed, rather than poking HW, so PF and VF instances present the
same view.
This lets IGT and other tools query the PAT configuration without
carrying platform-specific tables or mirroring kernel logic.
v2: (Jonathan)
- Only append "(* = reserved entry)" to the PAT table header on Xe2+
platforms where it actually applies.
- Deduplicate the PTA/ATS mode printing by introducing the small
drm_printf_pat_mode() helper macro.
v3: (Matt)
- Print IDX[XE_CACHE_NONE_COMPRESSION] on every Xe2+ platform so the
dump always reflects the value the driver might use (even if it defaults
to 0) and future IP revisions don’t need extra condition tweaks.
v4:
- Drop the drm_printf_pat_mode macro and introduce a real helper
xe2_pat_entry_dump(). (Jani)
- Reuse the helper across all PTA/ATS/PAT dumps for xe2+ entries to keep
output format identical.
v5: (Matt)
- Split the original patch into two: one for refactoring helpers, one for
the new debugfs entry.
CC: Jani Nikula <jani.nikula@intel.com>
Suggested-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Xin Wang <x.wang@intel.com>
Reviewed-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patch.msgid.link/20251205070633.28072-1-x.wang@intel.com
Move the PAT entry formatting into shared helper functions to ensure
consistency and enable code reuse.
This preparation is necessary for a follow-up patch that introduces a
software-based PAT dump, which is required for debugging on VFs where
hardware access is limited.
V2: (Matt)
- Xe3p XPC doesn’t define COMP_EN; omit it to match bspec and avoid
confusion.
Suggested-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Xin Wang <x.wang@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patch.msgid.link/20251205070220.27859-1-x.wang@intel.com
Enhance the PAT table dump by marking reserved entries with an
asterisk (*) for improved readability and debugging.
V2:
Added a note in the "PAT table" header explaining the meaning of
the asterisk(*) to improve clarity for readers. (Matt Roper)
V3:
Introduced a valid field in struct xe_pat_table_entry to
explicitly track whether an entry is valid or reserved, avoiding
reliance on coh_mode == 0. (Matt Roper)
Signed-off-by: Xin Wang <x.wang@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patch.msgid.link/20251030221734.1058350-1-x.wang@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Now that the PAT settings for the new special entries introduced by Xe2
are decided during early software init and left NULL on platforms they
don't apply to, there's no need to keep separate programming functions
for pre-Xe2 and post-Xe2 platforms. Consolidate down to a single pair
of programming functions (mcr and non-mcr) that can be used on any
platform.
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Link: https://lore.kernel.org/r/20250613214751.792066-4-matthew.d.roper@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Decide whether programming of the special ATS and PTA PAT entries is
necessary (and which entries should be programmed) during early software
initialization rather than hardcoding this into the 'program' functions.
Future platforms may want to re-use the same functions but utilize
different special entry values. Consolidating all of the decisions
into one place keeps things simple.
Reviewed-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
Link: https://lore.kernel.org/r/20250613214751.792066-3-matthew.d.roper@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Dealing with CCS state is significant on LNL+, where we end up clearing
the compression state on every page alloc using the blitter for user
buffers, including also saving and restoring it when moving between
domains, plus we need to alloc extra pages to hold the raw CCS state for
the save step.
However all compression PAT modes, on platforms like LNL, also require
coh_none, meaning that only WC memory can use compression in the first
place. With this we can be sneaky and completely ignore CCS for WB
buffers, which is likely the common case anyway. This would then skip
all blitter moves/clears between sys <-> tt and then also means we can
drop the extra CCS pages.
This should be safe since there is no way to interact with the
compression state (potentially uncleared) without using a PAT enabled
index (which is rejected at bind), including if trying to be malicious
and copy the raw CCS state from userpace, which should give back all
zeroes if the src surface (indirect) is lacking compressed PAT index.
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Link: https://lore.kernel.org/r/20250516153810.223530-2-matthew.auld@intel.com
xe_force_wake_get() now returns the reference count-incremented domain
mask. If it fails for individual domains, the return value will always
be 0. However, for XE_FORCEWAKE_ALL, it may return a non-zero value even
in the event of failure. Update the return handling of xe_force_wake_get()
to reflect this behavior, and ensure that the return value is passed as
input to xe_force_wake_put().
v3
- return xe_wakeref_t instead of int in xe_force_wake_get()
- xe_force_wake_put() error doesn't need to be checked. It internally
WARNS on domain ack failure.
- don't use xe_assert() to report HW errors (Michal)
v5
- return unsigned int from xe_force_wake_get()
- remove redundant warns
v7
- Fix commit message
- Remove redundant header
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Nirmoy Das <nirmoy.das@intel.com>
Reviewed-by: Badal Nilawar <badal.nilawar@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20241014075601.2324382-20-himal.prasad.ghimiray@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Future uapi needs to give userspace the ability to select the pat_index
for a given vm_bind. However we need to be able to extract the coherency
mode from the provided pat_index to ensure it's compatible with the
cpu_caching mode set at object creation. There are various security
reasons for why this matters. However the pat_index itself is very
platform specific, so seems reasonable to annotate each platform
definition of the pat table. On some older platforms there is no
explicit coherency mode, so we just pick whatever makes sense.
v2:
- Simplify with COH_AT_LEAST_1_WAY
- Add some kernel-doc
v3 (Matt Roper):
- Some small tweaks
v4:
- Rebase
v5:
- Rebase on Xe2 PAT additions
v6:
- Rebase on removal of coh_mode from uapi
Bspec: 45101, 44235 #xe
Bspec: 70552, 71582, 59400 #xe2
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Pallavi Mishra <pallavi.mishra@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: José Roberto de Souza <jose.souza@intel.com>
Cc: Filip Hazubski <filip.hazubski@intel.com>
Cc: Carl Zhang <carl.zhang@intel.com>
Cc: Effie Yu <effie.yu@intel.com>
Cc: Zhengguo Xu <zhengguo.xu@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Pallavi Mishra <pallavi.mishra@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
This is useful to debug cache issues, to double check if the PAT
indexes match what they were supposed to be set to from spec.
v2: Add separate functions for XeHP, XeHPC and XeLPG so it correctly
reads the index based on MCR/REG registers and also decodes the
fields (Matt Roper)
v3: Starting with XeHPC, do not translate values to human-readable
formats as the main goal is to make it easy to compare the table
with the spec. Also, share a single array for xelp/xehp str map
(Matt Roper)
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://lore.kernel.org/r/20231006182325.3617685-3-lucas.demarchi@intel.com
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
These should replace the _MMIO() and MCR_REG() from i915, with the goal
of being more extensible, allowing to pass the additional fields for
struct xe_reg and struct xe_reg_mcr. Replace all uses of _MMIO() and
MCR_REG() in xe.
Since the RTP, reg-save-restore and WA infra are not ready to use the
new type, just undef the macro like was done for the i915 types
previously. That conversion will come later.
v2: Remove MEDIA_SOFT_SCRATCH_COUNT/MEDIA_SOFT_SCRATCH re-added by
mistake (Matt Roper)
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://lore.kernel.org/r/20230427223256.1432787-8-lucas.demarchi@intel.com
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Clarify a few things related to the PAT programming, particularly on
MTL:
- The register type doesn't change depending on the GT - what
happens is that media GT writes to other set of registers that
are not MCR
- Remove "UNICAST": otherwise it's confusing why it's not using
MCR registers with the unicast function variant
Also, there isn't much reason to keep those parts as macros: promote
them to proper functions and let the compiler inline if it sees fit.
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://lore.kernel.org/r/20230427223256.1432787-5-lucas.demarchi@intel.com
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Starting with MTL, the number of entries in the PAT table increased to
16. The register offset jumped between index 7 and index 8, so a slight
adjustment is needed to ensure the PAT_INDEX macros select the proper
offset for the upper half of the table.
Note that although there are 16 registers in the hardware, the driver is
currently only asked to program the first 5, and we leave the rest at
their hardware default values. That means we don't actually touch the
upper half of the PAT table in the driver today and this patch won't
have any functional effect [yet].
Bspec: 44235
Reviewed-by: Nirmoy Das <nirmoy.das@intel.com>
Link: https://lore.kernel.org/r/20230324210415.2434992-7-matthew.d.roper@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
The PAT_INDEX registers are MCR registers on some platforms and unicast
on others. On MTL the handling even varies between GTs: the primary GT
uses MCR registers while the media GT uses unicast registers. Let's add
proper MCR programming on the relevant platforms/GTs.
Given that we PAT tables to change pretty regularly on future platforms,
we'll make PAT programming an exception to the usual model of assuming
new platforms should inherit the previous platform's behavior. Instead
we'll raise a warning if the current platform isn't handled in the
if/else ladder. This should help prevent subtle cache misbehavior if we
forget to add the table for a new platform.
Bspec: 66534, 67609, 67788
Reviewed-by: Nirmoy Das <nirmoy.das@intel.com>
Link: https://lore.kernel.org/r/20230324210415.2434992-4-matthew.d.roper@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
