Nicholas Kazlauskas ca8e2084e5 drm/amd/display: Use u16 for drm_bpp in DSC calculations ...

[Why]
DSC calculations fail because the u16 bits_per_pixel from
the DRM struct is being casted to the u8 drm_bpp parameters
and locals. Integer wraparound is happening because this
value is greater than 255.

[How]
Use u16 to match what's in the structure instead of u8.

Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Reviewed-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Acked-by: Qingqing Zhuo <qingqing.zhuo@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

2020-07-01 01:59:20 -04:00

..

dc_dsc.c

drm/amd/display: Rework dsc to isolate FPU operations

2020-07-01 01:59:20 -04:00

dscc_types.h

drm/amd/display: Drop CONFIG_DRM_AMD_DC_DCN2_0 and DSC_SUPPORTED

2019-11-13 15:29:44 -05:00

Makefile

drm/amd/display: Rework dsc to isolate FPU operations

2020-07-01 01:59:20 -04:00

qp_tables.h

drm/amd/display: Drop CONFIG_DRM_AMD_DC_DCN2_0 and DSC_SUPPORTED

2019-11-13 15:29:44 -05:00

rc_calc_dpi.c

drm/amd/display: Rework dsc to isolate FPU operations

2020-07-01 01:59:20 -04:00

rc_calc.c

drm/amd/display: Use u16 for drm_bpp in DSC calculations

2020-07-01 01:59:20 -04:00

rc_calc.h

drm/amd/display: Rework dsc to isolate FPU operations

2020-07-01 01:59:20 -04:00