Pull media updates from Mauro Carvalho Chehab:
- DVB budget legacy API was finally documented. It took only 20+ years
to get some documentation about it...
- hantro driver has gained support for STM32MP25 VDEC/VENC
- rkisp1 has gained support for i.MX8MP
- atomisp got rid of two items from its todo list. Still 5 items
pending for moving it out of staging
- lots of driver fixes, cleanups and improvements
* tag 'media/v6.9-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media: (252 commits)
media: rcar-isp: Disallow unbind of devices
media: usbtv: Remove useless locks in usbtv_video_free()
media: mediatek: vcodec: avoid -Wcast-function-type-strict warning
media: ttpci: fix two memleaks in budget_av_attach
media: go7007: fix a memleak in go7007_load_encoder
media: dvb-frontends: avoid stack overflow warnings with clang
media: pvrusb2: fix uaf in pvr2_context_set_notify
media: usb: s2255: Refactor s2255_get_fx2fw
media: ti: j721e-csi2rx: Convert to platform remove callback returning void
media: stm32-dcmipp: Convert to platform remove callback returning void
media: nxp: imx8-isi: Convert to platform remove callback returning void
media: nuvoton: Convert to platform remove callback returning void
media: chips-media: wave5: Convert to platform remove callback returning void
media: chips-media: wave5: Remove unnecessary semicolons
media: i2c: imx290: Fix IMX920 typo
media: platform: replace of_graph_get_next_endpoint()
media: i2c: replace of_graph_get_next_endpoint()
media: ivsc: csi: Make use of sub-device state
media: ivsc: csi: Swap SINK and SOURCE pads
media: ipu-bridge: Serialise calls to IPU bridge init
...
The i.MX8MP has extra register fields in the memory interface control
register for setting the output format, which work with the output
alignment format register for byte-swapping and LSB/MSB alignment.
With processed and 8-bit raw streams, it doesn't cause any problems to
not set these, but with raw streams of higher bit depth the endianness
is swapped and the data is not aligned properly.
Add support for setting these registers and plumb them in to fix this.
While at it, reflow a comment related to the forced configuration
update.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Add support for UYVY as an output format. The uv_swap bit in the
MI_XTD_FORMAT_CTRL register that is used for the NV formats does not
work for packed YUV formats. Thus, UYVY support is implemented via
byte-swapping. This method clearly does not work for implementing
support for YVYU and VYUY.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Tested-by: Adam Ford <aford173@gmail.com>
The ISP version in the i.MX8MP has an MI_OUTPUT_ALIGN_FORMAT register
that the rk3399 does not have. This register allows swapping bytes,
which can be used to implement UYVY from YUYV.
Add a flag to the format info in the list of formats supported by the
capture v4l2 devices, and update enum_fmt and s_fmt to take it into
account.
To signify the presence of this feature, reuse the MAIN_STRIDE feature
flag, as it is very likely that any ISP version that supports one of
these two features will also support the other.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Tested-by: Adam Ford <aford173@gmail.com>
Some versions of the ISP supported by the rkisp1 driver, such as the ISP
in the i.MX8MP, lack the dual crop registers and don't support cropping
at the resizer input. They instead rely on cropping in the Image
Stabilization module, at the output of the ISP, to modify the resizer
input size and implement digital zoom.
Add a dual crop feature flag to distinguish the versions of the ISP that
support dual crop from those that don't, and make sure that the sink
crop is set to the sink format when dual crop is not supported.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Tested-by: Adam Ford <aford173@gmail.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
The driver requests the interrupts as IRQF_SHARED, so the interrupt
handlers can be called at any time. If such a call happens while the ISP
is powered down, the SoC will hang as the driver tries to access the
ISP registers.
This can be reproduced even without the platform sharing the IRQ line:
Enable CONFIG_DEBUG_SHIRQ and unload the driver, and the board will
hang.
Fix this by adding a new field, 'irqs_enabled', which is used to bail
out from the interrupt handler when the ISP is not operational.
Link: https://lore.kernel.org/r/20231218-rkisp-shirq-fix-v1-2-173007628248@ideasonboard.com
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Rename min_buffers_needed into min_queued_buffers and update
the documentation about it.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
[hverkuil: Drop the change where min_queued_buffers + 1 buffers would be]
[hverkuil: allocated. Now this patch only renames this field instead of making]
[hverkuil: a functional change as well.]
[hverkuil: Renamed 3 remaining min_buffers_needed occurrences.]
Supported media bus codes on the resizer sink pad are identical to the
ISP source pad. The .enum_mbus_code() handler thus delegates the
enumeration to the ISP's operation. This is problematic for two
reasons:
- Format enumeration on the ISP source pad is dependent on the format
configured on the ISP sink pad for the same subdev state (TRY or
ACTIVE), while format enumeration on the resizer sink pad should
return all formats supported by the resizer subdev, regardless of the
ISP configuration.
- Delegating the operation involves creating a fake v4l2_subdev_state on
the stack to pass to the ISP .enum_mbus_code() handler. This gets in
the way of evolution of both the ISP enumeration handler and, more
generally, the V4L2 subdev state infrastructure.
Fix those two issues by implementing format enumeration manually for the
resizer.
Link: https://lore.kernel.org/r/20231126020948.2700-1-laurent.pinchart@ideasonboard.com
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
In rkisp1_isp_stop() and rkisp1_csi_disable() the driver masks the
interrupts and then apparently assumes that the interrupt handler won't
be running, and proceeds in the stop procedure. This is not the case, as
the interrupt handler can already be running, which would lead to the
ISP being disabled while the interrupt handler handling a captured
frame.
This brings up two issues: 1) the ISP could be powered off while the
interrupt handler is still running and accessing registers, leading to
board lockup, and 2) the interrupt handler code and the code that
disables the streaming might do things that conflict.
It is not clear to me if 2) causes a real issue, but 1) can be seen with
a suitable delay (or printk in my case) in the interrupt handler,
leading to board lockup.
Link: https://lore.kernel.org/r/20231207-rkisp-irq-fix-v3-4-358a2c871a3c@ideasonboard.com
Tested-by: Adam Ford <aford173@gmail.com> #imx8mp-beacon
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Add support for the multi-planar variants of NV12. The RGA is now able
to exchange DMABUFs with other devices that only support multi-planar
NV12, for example the Hantro JPEG encoder.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Multi-planar formats may have multiple planes that must be handled and
correctly mapped into a continuous buffer for the RGA by using the DMA
descriptors.
The plane offsets in the continuous mapping may now start at page
boundaries and the previous calculation based on the frame sizes is only
valid for planar buffers in a single memory. Therefore, the offsets must
be detected and set while creating the mapping.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Switch to the multi-planar API, which allows to handle buffers with
separate planes.
The RGA driver doesn't expose multi-planar formats, yet. The existing
contiguous planar formats can be used with the multi-planar API as well,
but the multi-planar API is required for multi-planar formats.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Use the provided V4L2_TYPE_IS_{OUTPUT,CAPTURE} macros to check if the
buffer or queue is OUTPUT or CAPTURE. The macros work also work for the
_MPLANE buffer and queue types and make it easier to switch to the
multi-planar API.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The local variable allows to simplify the accessed to the format and
makes it easier to change the type of the format.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Use the pixelformat instead of the v4l2_format to find the rga_fmt. This
avoids knowing the structure and type of v4l2_format in rga_fmt_find and
simplifies the function.
Also cleanup the users of the function. In try_fmt always return the
found pixel format to make sure that the pixel format is always set.
Thus, we can be sure that we will find the rga_fmt in s_fmt and can drop
the check if a given format has been found.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The try_fmt should limit the width and height to the know limits of the
RGA. Use the clamp() helper instead of open coding the clamping.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The RGA DMA descriptor list contains only 32-bit addresses. Set the
dma_mask to only allocate memory that is addressable by the descriptors.
This prevents errors when preparing vb2 buffers that were allocated by
the RGA.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Calculate the plane offsets and store them with the video buffer while
creating the buffer mapping.
This allows the driver to more freely handle the memory of the DMA
mapping as the offsets and the mapping can be kept in sync.
The driver still has to update the offsets to respect the configured
cropping and rotation, but this calculation is now separated from the
calculation of the plane offsets.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Split the register setup for the source and destination video buffers
into separate functions.
This is a cleanup to make the code more readable by separating the
offset calculation for the different buffers and prepares the driver for
using pre-calculated offsets of planes.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The RGA driver allocates two buffers for the DMA descriptors of the
input and output buffers. Whenever a new job is processed, the
descriptor list is updated for the current buffers.
By updating the descriptor list during buf_prepare, it is possible to
correctly fail DMABUF imports if the buffers that shall be imported are
not within the 32 bit address range that can be addressed by the RGA.
Managing the DMA descriptor list with the buffer also makes it easier to
track the buffer mapping and the plane offsets into this mapping.
The cost is that the driver now requires DMA coherent memory per buffer
for the descriptor list. However, the size scales with the size of the
video buffers and is not allocated if the RGA is not used.
While at it, use dma_alloc_coherent to allocate the descriptors and get
rid of the virt_to_phys calls to get the physical addresses.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The IOMMU of the RGA is programmed with a list of DMA descriptors that
contain an 32 bit address per 4k page in the video buffers. The address
in the descriptor points to the start address of the page.
Introduce 'struct rga_dma_desc' to make the handling of the DMA
descriptors explicit instead of hiding them behind standard types.
While at it, use provided helpers for iterating the sg_table instead of
manually calculating the DMA addresses.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
When using 32 bit RGB formats, the RGA on the rk3568 produces wrong
colors as the wrong color channels are read or written. The reason is
that the format description for the channel swizzeling is wrong and the
wrong bits are configured. For example, when converting ARGB32 to NV12,
the alpha channel is used as blue channel.. This doesn't happen if the
color format is the same on both sides.
Fix the color_swap settings of the formats to correctly handle 32 bit
RGB formats.
For RGA_COLOR_FMT_XBGR8888, the RGA_COLOR_ALPHA_SWAP bit doesn't have an
effect. Thus, it isn't possible to handle the V4L2_PIX_FMT_XRGB32. Thus,
it is removed from the list of supported formats.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The subdev .init_cfg() operation is affected by two issues:
- It has long been extended to initialize a whole v4l2_subdev_state
instead of just a v4l2_subdev_pad_config, but its name has stuck
around.
- Despite operating on a whole subdev state and not being directly
exposed to the subdev users (either in-kernel or through the userspace
API), .init_cfg() is categorized as a subdev pad operation.
This participates in making the subdev API confusing for new developers.
Fix it by renaming the operation to .init_state(), and make it a subdev
internal operation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Michael Riesch <michael.riesch@wolfvision.net> # for imx415
Acked-by: Shuah Khan <skhan@linuxfoundation.org> # for vimc
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
[Sakari Ailus: Resolved a conflict in Renesas vsp1 driver.]
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The resizer is used to scale the image, but also to change the
subsampling of YUV formats. Both the luma and chroma dimensions need to
be taken into account to decide whether or not to enable the resizer.
The current implementation disables the resizer if the chroma vertical
size isn't changed, which would be the case when scaling up by a factor
of 2 vertically while at the same time converting from YUV 4:2:2 to
4:2:0. Fix it by checking the luma sizes too.
While at it, reflow and clarify comments in the function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The debug messages that show the resizer configuration are only printed
if the driver enables the resizer. This prevents checking the resizer
configuration when the driver believes it should be disabled. Fix it by
moving the dev_dbg() statements earlier.
Also, combine the two debug prints into a single one to gather all the
information in one place, which makes reading the log easier.
While at it, use %u instead of %d to print unsigned values.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The rkisp1_rsz_config() and rkisp1_rsz_config_regs() functions use a
v4l2_rect to pass frame sizes, leaving the top and left members unused
and uninitialized. Use v4l2_area instead.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The self path has a Y_PIC_SIZE register that needs to be programmed to
the total number of pixels, including the stride. This isn't done by the
driver, fix it.
While at it, reorder the register write order to sort them by address.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The dual crop register is the same for both the MP and SP channels. Drop
it from the rkisp1_rsz_config structure and use the
RKISP1_CIF_DUAL_CROP_CTRL macro directly in the code.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The line stride is expressed in the hardware as a number of pixels for
the first plane. The bytesperline must thus be a multiple of the first
plane's bpp value. Enforce this constraint.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Hex constants in the media subsystem are typically in lowercase. The
rkisp1 driver mostly follows that convention already, except in the
register definitions. Convert all hex constants to lowercase for
consistency.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Use the V4L2 subdev active state API to store the active format and crop
rectangle. This simplifies the driver not only by dropping the state
stored in the rkisp1_csi structure, but also by replacing the ops_lock
with the state lock.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
