Some versions of the ISP supported by the rkisp1 driver, such as the ISP
in the i.MX8MP, lack the dual crop registers and don't support cropping
at the resizer input. They instead rely on cropping in the Image
Stabilization module, at the output of the ISP, to modify the resizer
input size and implement digital zoom.
Add a dual crop feature flag to distinguish the versions of the ISP that
support dual crop from those that don't, and make sure that the sink
crop is set to the sink format when dual crop is not supported.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Tested-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Tested-by: Adam Ford <aford173@gmail.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Supported media bus codes on the resizer sink pad are identical to the
ISP source pad. The .enum_mbus_code() handler thus delegates the
enumeration to the ISP's operation. This is problematic for two
reasons:
- Format enumeration on the ISP source pad is dependent on the format
configured on the ISP sink pad for the same subdev state (TRY or
ACTIVE), while format enumeration on the resizer sink pad should
return all formats supported by the resizer subdev, regardless of the
ISP configuration.
- Delegating the operation involves creating a fake v4l2_subdev_state on
the stack to pass to the ISP .enum_mbus_code() handler. This gets in
the way of evolution of both the ISP enumeration handler and, more
generally, the V4L2 subdev state infrastructure.
Fix those two issues by implementing format enumeration manually for the
resizer.
Link: https://lore.kernel.org/r/20231126020948.2700-1-laurent.pinchart@ideasonboard.com
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
The subdev .init_cfg() operation is affected by two issues:
- It has long been extended to initialize a whole v4l2_subdev_state
instead of just a v4l2_subdev_pad_config, but its name has stuck
around.
- Despite operating on a whole subdev state and not being directly
exposed to the subdev users (either in-kernel or through the userspace
API), .init_cfg() is categorized as a subdev pad operation.
This participates in making the subdev API confusing for new developers.
Fix it by renaming the operation to .init_state(), and make it a subdev
internal operation.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Michael Riesch <michael.riesch@wolfvision.net> # for imx415
Acked-by: Shuah Khan <skhan@linuxfoundation.org> # for vimc
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
[Sakari Ailus: Resolved a conflict in Renesas vsp1 driver.]
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The resizer is used to scale the image, but also to change the
subsampling of YUV formats. Both the luma and chroma dimensions need to
be taken into account to decide whether or not to enable the resizer.
The current implementation disables the resizer if the chroma vertical
size isn't changed, which would be the case when scaling up by a factor
of 2 vertically while at the same time converting from YUV 4:2:2 to
4:2:0. Fix it by checking the luma sizes too.
While at it, reflow and clarify comments in the function.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The debug messages that show the resizer configuration are only printed
if the driver enables the resizer. This prevents checking the resizer
configuration when the driver believes it should be disabled. Fix it by
moving the dev_dbg() statements earlier.
Also, combine the two debug prints into a single one to gather all the
information in one place, which makes reading the log easier.
While at it, use %u instead of %d to print unsigned values.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The rkisp1_rsz_config() and rkisp1_rsz_config_regs() functions use a
v4l2_rect to pass frame sizes, leaving the top and left members unused
and uninitialized. Use v4l2_area instead.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The dual crop register is the same for both the MP and SP channels. Drop
it from the rkisp1_rsz_config structure and use the
RKISP1_CIF_DUAL_CROP_CTRL macro directly in the code.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Use the V4L2 subdev active state API to store the active format and crop
rectangle. This simplifies the driver not only by dropping the state
stored in the rkisp1_resizer structure, but also by replacing the
ops_lock with the state lock.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Commit 0d346d2a6f ("media: v4l2-subdev: add subdev-wide state struct")
applied a large media tree-wide change produced by coccinelle. It was so
large that a set of identical indentation issues went unnoticed during
review. Fix them.
While at it, and because it's easy to review both changes together, add
a trailing comma for the last (and only) struct member initialization of
the related structures, to avoid future changes should new fields need
to be initialized.
Fixes: 0d346d2a6f ("media: v4l2-subdev: add subdev-wide state struct")
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
The resizer doesn't deal with color spaces, so it can accept any color
space on its input, and propagates it unchanged to its output. When
operating with a Bayer input format (in pass-through mode) further
restrict the YCbCr encoding and quantization to Rec 601 and full range
respectively, as for raw data the former ought to be ignored and the
latter is always full range.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Initialize the four color space fields on the sink and source video pads
of the resizer in the .init_cfg() operation. The resizer can't perform
any color space conversion, so set the sink and source color spaces to
the same defaults, which match the ISP source video pad default.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
The upcoming CSI receiver split from the ISP to a separate source file
will need to be able to access the list of formats supported by the
driver. Move it out of the ISP's header and into the common header, and
add helper functions for accessing it so that the format list doesn't
need to be stored in the header.
Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
The debugfs support already includes support to dump core and ISP
registers. Move the resizer register dump there too to make the
userspace interface consistent.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
The registers for the mainpath and selfpath resizers are located at the
same offset from the instance-specific base. Use this to simplify
register access, removing the need to store per-register offsets in the
rkisp1_rsz_config structure.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
While writel() takes the value and address arguments in that order, most
write functions (including in the regmap API) use the opposite
convention. Having the value first is considered confusing, and often
leads to more difficult to read code compared to the opposite convention
where the write call and the register name often fit on a single line:
rkisp1_write(rkisp1, RKISP1_CIF_THE_REG_NAME,
complicate_calculation + for / the_register
value + goes | here);
Swap the arguments of the rkisp1_write() function, and use the following
semantic patch to update the callers:
@@
expression rkisp1, value, address;
@@
- rkisp1_write(rkisp1, value, address)
+ rkisp1_write(rkisp1, address, value)
This commit also includes a few additional line break cleanups in the
rkisp1_write() calls, but no other manual change.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
The rkisp1_rsz_register() and rkisp1_rsz_unregister() functions don't
destroy the mutex (in the error path for the former). Fix this, simplify
error handling at registration time as media_entity_cleanup() can be
called on an uninitialized entity, and make rkisp1_rsz_unregister() and
rkisp1_resizer_devs_unregister() safe to be called on an unregistered
resizer subdev to prepare for simplification of error handling at probe
time.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
We have 'struct v4l2_subdev_pad_config' which contains configuration for
a single pad used for the TRY functionality, and an array of those
structs is passed to various v4l2_subdev_pad_ops.
I was working on subdev internal routing between pads, and realized that
there's no way to add TRY functionality for routes, which is not pad
specific configuration. Adding a separate struct for try-route config
wouldn't work either, as e.g. set-fmt needs to know the try-route
configuration to propagate the settings.
This patch adds a new struct, 'struct v4l2_subdev_state' (which at the
moment only contains the v4l2_subdev_pad_config array) and the new
struct is used in most of the places where v4l2_subdev_pad_config was
used. All v4l2_subdev_pad_ops functions taking v4l2_subdev_pad_config
are changed to instead take v4l2_subdev_state.
The changes to drivers/media/v4l2-core/v4l2-subdev.c and
include/media/v4l2-subdev.h were written by hand, and all the driver
changes were done with the semantic patch below. The spatch needs to be
applied to a select list of directories. I used the following shell
commands to apply the spatch:
dirs="drivers/media/i2c drivers/media/platform drivers/media/usb drivers/media/test-drivers/vimc drivers/media/pci drivers/staging/media"
for dir in $dirs; do spatch -j8 --dir --include-headers --no-show-diff --in-place --sp-file v4l2-subdev-state.cocci $dir; done
Note that Coccinelle chokes on a few drivers (gcc extensions?). With
minor changes we can make Coccinelle run fine, and these changes can be
reverted after spatch. The diff for these changes is:
For drivers/media/i2c/s5k5baf.c:
@@ -1481,7 +1481,7 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd,
&s5k5baf_cis_rect,
v4l2_subdev_get_try_crop(sd, cfg, PAD_CIS),
v4l2_subdev_get_try_compose(sd, cfg, PAD_CIS),
- v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT)
+ v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT),
};
s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r);
return 0;
For drivers/media/platform/s3c-camif/camif-capture.c:
@@ -1230,7 +1230,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd,
*mf = camif->mbus_fmt;
break;
- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* crop rectangle at camera interface input */
mf->width = camif->camif_crop.width;
mf->height = camif->camif_crop.height;
@@ -1332,7 +1332,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
}
break;
- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* Pixel format can be only changed on the sink pad. */
mf->code = camif->mbus_fmt.code;
mf->width = crop->width;
The semantic patch is:
// <smpl>
// Change function parameter
@@
identifier func;
identifier cfg;
@@
func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...)
{
<...
- cfg
+ sd_state
...>
}
// Change function declaration parameter
@@
identifier func;
identifier cfg;
type T;
@@
T func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...);
// Change function return value
@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...)
{
...
}
// Change function declaration return value
@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...);
// Some drivers pass a local pad_cfg for a single pad to a called function. Wrap it
// inside a pad_state.
@@
identifier func;
identifier pad_cfg;
@@
func(...)
{
...
struct v4l2_subdev_pad_config pad_cfg;
+ struct v4l2_subdev_state pad_state = { .pads = &pad_cfg };
<+...
(
v4l2_subdev_call
|
sensor_call
|
isi_try_fse
|
isc_try_fse
|
saa_call_all
)
(...,
- &pad_cfg
+ &pad_state
,...)
...+>
}
// If the function uses fields from pad_config, access via state->pads
@@
identifier func;
identifier state;
@@
func(...,
struct v4l2_subdev_state *state
, ...)
{
<...
(
- state->try_fmt
+ state->pads->try_fmt
|
- state->try_crop
+ state->pads->try_crop
|
- state->try_compose
+ state->pads->try_compose
)
...>
}
// If the function accesses the filehandle, use fh->state instead
@@
struct v4l2_subdev_fh *fh;
@@
- fh->pad
+ fh->state
@@
struct v4l2_subdev_fh fh;
@@
- fh.pad
+ fh.state
// Start of vsp1 specific
@@
@@
struct vsp1_entity {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
};
@@
symbol entity;
@@
vsp1_entity_init(...)
{
...
entity->config =
- v4l2_subdev_alloc_pad_config
+ v4l2_subdev_alloc_state
(&entity->subdev);
...
}
@@
symbol entity;
@@
vsp1_entity_destroy(...)
{
...
- v4l2_subdev_free_pad_config
+ v4l2_subdev_free_state
(entity->config);
...
}
@exists@
identifier func =~ "(^vsp1.*)|(hsit_set_format)|(sru_enum_frame_size)|(sru_set_format)|(uif_get_selection)|(uif_set_selection)|(uds_enum_frame_size)|(uds_set_format)|(brx_set_format)|(brx_get_selection)|(histo_get_selection)|(histo_set_selection)|(brx_set_selection)";
symbol config;
@@
func(...) {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
}
// End of vsp1 specific
// Start of rcar specific
@@
identifier sd;
identifier pad_cfg;
@@
rvin_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}
// End of rcar specific
// Start of rockchip specific
@@
identifier func =~ "(rkisp1_rsz_get_pad_fmt)|(rkisp1_rsz_get_pad_crop)|(rkisp1_rsz_register)";
symbol rsz;
symbol pad_cfg;
@@
func(...)
{
+ struct v4l2_subdev_state state = { .pads = rsz->pad_cfg };
...
- rsz->pad_cfg
+ &state
...
}
@@
identifier func =~ "(rkisp1_isp_get_pad_fmt)|(rkisp1_isp_get_pad_crop)";
symbol isp;
symbol pad_cfg;
@@
func(...)
{
+ struct v4l2_subdev_state state = { .pads = isp->pad_cfg };
...
- isp->pad_cfg
+ &state
...
}
@@
symbol rkisp1;
symbol isp;
symbol pad_cfg;
@@
rkisp1_isp_register(...)
{
+ struct v4l2_subdev_state state = { .pads = rkisp1->isp.pad_cfg };
...
- rkisp1->isp.pad_cfg
+ &state
...
}
// End of rockchip specific
// Start of tegra-video specific
@@
identifier sd;
identifier pad_cfg;
@@
__tegra_channel_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}
@@
identifier sd_state;
@@
__tegra_channel_try_format(...)
{
...
struct v4l2_subdev_state *sd_state;
<...
- sd_state->try_crop
+ sd_state->pads->try_crop
...>
}
// End of tegra-video specific
// </smpl>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
When setting the source media bus code in the resizer,
we first check that the current media bus code in the
source is yuv encoded format. This is done by
retrieving the data from the formats list of the isp
entity. This cause a crash when the media bus code on the
source is YUYV8_1_5X8 which is not supported by the isp
entity. Instead we should test the sink format of the resizer
which is guaranteed to be supported by the isp entity.
Fixes: 251b6eebb6 ("media: staging: rkisp1: rsz: Add support to more YUV encoded mbus codes on src pad")
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Acked-by: Helen Koike <helen.koike@collabora.com>
Tested-by: Sebastian Fricke <sebastian.fricke.linux@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
All the items in the TODO list were addressed, uapi was reviewed,
documentation written, checkpatch errors fixed, several bugs fixed.
There is no big reason to keep this driver in staging, so move it out.
Dt-bindings Verified with:
make ARCH=arm64 dt_binding_check DT_SCHEMA_FILES=Documentation/devicetree/bindings/media/rockchip-isp1.yaml
Fields of MAINTAINERS file sorted according to output of
./scripts/parse-maintainers.pl --input=MAINTAINERS --output=MAINTAINERS
--order
[dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging]
[dt-bindings: media: rkisp1: move rockchip-isp1 bindings out of staging]
[hverkuil: fix various checkpatch alignment warnings]
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Acked-by: Rob Herring <robh@kernel.org>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
