bybrooklyn/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-05-05 15:02:40 -04:00
Code Issues Packages Projects Releases Wiki Activity

drm/virtio: fix DRM_FORMAT_* handling

Browse Source 
Use DRM_FORMAT_HOST_XRGB8888, so we are using the correct format code
on bigendian machines.  Also set the quirk_addfb_prefer_host_byte_order
mode_config bit so drm_mode_addfb() asks for the correct format code.

Both DRM_FORMAT_* and VIRTIO_GPU_FORMAT_* are defined to be little
endian, so using a different mapping on bigendian machines is wrong.
It's there because of broken drm_mode_addfb() behavior.  So with
drm_mode_addfb() being fixed we can fix this too.

While wading through the code I've noticed we have a little issue in
virtio:  We attach a format to the bo when it is created
(DRM_IOCTL_MODE_CREATE_DUMB), not when we map it as framebuffer
(DRM_IOCTL_MODE_ADDFB).  Easy way out:  Support a single format only.
Pick DRM_FORMAT_HOST_XRGB8888, it is the only one actually used in
practice.  Drop unused mappings in virtio_gpu_translate_format().

With this patch applied both ADDFB and ADDFB2 ioctls work correctly in
the virtio-gpu.ko driver on big endian machines.  Without the patch only
ADDFB (which still seems to be used by the majority of userspace) works
correctly.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20180921134704.12826-6-kraxel@redhat.com
This commit is contained in:
Gerd Hoffmann
2018-09-21 15:47:03 +02:00
parent 86351de023
commit 42fd9e6c29
4 changed files with 13 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
drivers/gpu/drm/virtio/virtgpu_plane.c
View File

@@ -28,22 +28,11 @@
#include <drm/drm_atomic_helper.h>
static const uint32_t virtio_gpu_formats[] = {
DRM_FORMAT_XRGB8888,
DRM_FORMAT_ARGB8888,
DRM_FORMAT_BGRX8888,
DRM_FORMAT_BGRA8888,
DRM_FORMAT_RGBX8888,
DRM_FORMAT_RGBA8888,
DRM_FORMAT_XBGR8888,
DRM_FORMAT_ABGR8888,
DRM_FORMAT_HOST_XRGB8888,
};
static const uint32_t virtio_gpu_cursor_formats[] = {
#ifdef __BIG_ENDIAN
DRM_FORMAT_BGRA8888,
#else
DRM_FORMAT_ARGB8888,
#endif
DRM_FORMAT_HOST_ARGB8888,
};
uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc)
@@ -51,32 +40,6 @@ uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc)
uint32_t format;
switch (drm_fourcc) {
#ifdef __BIG_ENDIAN
case DRM_FORMAT_XRGB8888:
format = VIRTIO_GPU_FORMAT_X8R8G8B8_UNORM;
break;
case DRM_FORMAT_ARGB8888:
format = VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM;
break;
case DRM_FORMAT_BGRX8888:
format = VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM;
break;
case DRM_FORMAT_BGRA8888:
format = VIRTIO_GPU_FORMAT_B8G8R8A8_UNORM;
break;
case DRM_FORMAT_RGBX8888:
format = VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM;
break;
case DRM_FORMAT_RGBA8888:
format = VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM;
break;
case DRM_FORMAT_XBGR8888:
format = VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM;
break;
case DRM_FORMAT_ABGR8888:
format = VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM;
break;
#else
case DRM_FORMAT_XRGB8888:
format = VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM;
break;
@@ -89,19 +52,6 @@ uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc)
case DRM_FORMAT_BGRA8888:
format = VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM;
break;
case DRM_FORMAT_RGBX8888:
format = VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM;
break;
case DRM_FORMAT_RGBA8888:
format = VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM;
break;
case DRM_FORMAT_XBGR8888:
format = VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM;
break;
case DRM_FORMAT_ABGR8888:
format = VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM;
break;
#endif
default:
/*
* This should not happen, we handle everything listed