When writing unittests for kdoc_output, it became clear that
the logic with handles a series of KdocItem symbols from
a single file belons to kdoc_output, and not to kdoc_files.
Move the code to it.
While here, also ensure that self.config will be placed
together with set.out_style.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <4ebc26e37a0b544c50d50b8077760f147fa6a535.1773823995.git.mchehab+huawei@kernel.org>
The Sphinx output from autodoc doesn't automatically break long
lines, except on spaces.
Change KernRe __repr__() to break the pattern on multiple strings,
each one with a maximum limit of 60 characters.
With that, documentation output for KernRe should now be displayable,
even on long strings.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <60c264a9d277fed655b1a62df2195562c8596090.1773823995.git.mchehab+huawei@kernel.org>
Most of the rules inside CTransforms are of the type CMatch.
Don't re-parse the source code every time.
Doing this doesn't change the output, but makes kdoc almost
as fast as before the tokenizer patches:
# Before tokenizer patches
$ time ./scripts/kernel-doc . -man >original 2>&1
real 0m42.933s
user 0m36.523s
sys 0m1.145s
# After tokenizer patches
$ time ./scripts/kernel-doc . -man >before 2>&1
real 1m29.853s
user 1m23.974s
sys 0m1.237s
# After this patch
$ time ./scripts/kernel-doc . -man >after 2>&1
real 0m48.579s
user 0m45.938s
sys 0m0.988s
$ diff -s before after
Files before and after are identical
Manually checked the differences between original and after
with:
$ diff -U0 -prBw original after|grep -v Warning|grep -v "@@"|less
They're due:
- whitespace fixes;
- struct_group are now better handled;
- several badly-generated man pages from broken inline kernel-doc
markups are now fixed.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <1cc2a4286ebf7d4b2d03fcaf42a1ba9fa09004b9.1773770483.git.mchehab+huawei@kernel.org>
Changeset 2b957decdb6c ("docs: kdoc: don't add broken comments inside prototypes")
revealed a hidden bug at split_struct_proto(): some comments there may break
its capability of properly identifying a struct.
Fixing it is as simple as stripping comments before calling it.
Fixes: 2b957decdb6c ("docs: kdoc: don't add broken comments inside prototypes")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <dcff37b6da5329aea415de31f543b6a1c2cbbbce.1773770483.git.mchehab+huawei@kernel.org>
The previous approach were to unwind nested structs/unions.
Now that we have a logic that can handle it well, use it to
ensure that struct_group macros will properly reflect the
actual struct.
Note that the replacemend logic still simplifies the code
a little bit, as the basic build block for struct group is:
union { \
struct { MEMBERS } ATTRS; \
struct __struct_group_tag(TAG) { MEMBERS } ATTRS NAME; \
} ATTRS
There:
- ATTRS is meant to add extra macro attributes like __packed
which we already discard, as they aren't relevant to
document struct members;
- TAG is used only when built with __cplusplus.
So, instead, convert them into just:
struct { MEMBERS };
Please notice that here, we're using the greedy version of the
backrefs, as MEMBERS is actually MEMBERS... on all such macros.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <24bf2c036b08814d9b4aabc27542fd3b2ff54424.1773770483.git.mchehab+huawei@kernel.org>
The NextMatch code is complex, and will become even more complex
if we add there support for arguments.
Now that we have a tokenizer, we can use a better solution,
easier to be understood.
Yet, to improve performance, it is better to make it use a
previously tokenized code, changing its ABI.
So, reimplement NextMatch using the CTokener class. Once it
is done, we can drop NestedMatch.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <fa818ea164216b17520b588e3f12b81499b76dd7.1773770483.git.mchehab+huawei@kernel.org>
Better handle comments inside structs. After those changes,
all unittests now pass:
test_private:
TestPublicPrivate:
test balanced_inner_private: OK
test balanced_non_greddy_private: OK
test balanced_private: OK
test no private: OK
test unbalanced_inner_private: OK
test unbalanced_private: OK
test unbalanced_struct_group_tagged_with_private: OK
test unbalanced_two_struct_group_tagged_first_with_private: OK
test unbalanced_without_end_of_line: OK
Ran 9 tests
This also solves a bug when handling STRUCT_GROUP() with a private
comment on it:
@@ -397134,7 +397134,7 @@ basic V4L2 device-level support.
unsigned int max_len;
unsigned int offset;
struct page_pool_params_slow slow;
- STRUCT_GROUP( struct net_device *netdev;
+ struct net_device *netdev;
unsigned int queue_idx;
unsigned int flags;
};
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <f83ee9e8c38407eaab6ad10d4ccf155fb36683cc.1773074166.git.mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <054763260f7b5459ad0738ed906d7c358d640692.1773770483.git.mchehab+huawei@kernel.org>
Parsing a file like drivers/scsi/isci/host.h, which contains
broken kernel-doc markups makes it create a prototype that contains
unmatched end comments.
That causes, for instance, struct sci_power_control to be shown this
this prototype:
struct sci_power_control {
* it is not. */ bool timer_started;
*/ struct sci_timer timer;
* requesters field. */ u8 phys_waiting;
*/ u8 phys_granted_power;
* mapped into requesters via struct sci_phy.phy_index */ struct isci_phy *requesters[SCI_MAX_PHYS];
};
as comments won't start with "/*" anymore.
Fix the logic to detect such cases, and keep adding the comments
inside it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <18e577dbbd538dcc22945ff139fe3638344e14f0.1773074166.git.mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <12ac4a97e2bd5a19d6537122c10098690c38d2c7.1773770483.git.mchehab+huawei@kernel.org>
While the main goal for kernel-doc is to be used inside the Linux
Kernel, other open source projects could benefit for it. That's
currently the case of QEMU, which has a fork, mainly due to two
reasons:
- they need an extra C function transform rule;
- they handle the html output a little bit different.
Add an extra optional argument to make easier for the code to be
shared, as, with that, QEMU can just create a new derivated class
that will contain its specific rulesets, and just copy the
remaining kernel-doc files as-is.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <6b274ddbdcd9d438c6848e00e410a2f65ef80ec2.1772810574.git.mchehab+huawei@kernel.org>
Drop context analysis and lock (tracking) attributes to avoid
kernel-doc warnings.
There are now lots of warnings like these:
Documentation/core-api/kref:328: ../include/linux/kref.h:72: WARNING: Invalid C declaration: Expected end of definition. [error at 96]
int kref_put_mutex (struct kref *kref, void (*release)(struct kref *kref), struct mutex *mutex) __cond_acquires(true# mutex)
------------------------------------------------------------------------------------------------^
Documentation/core-api/kref:328: ../include/linux/kref.h:94: WARNING: Invalid C declaration: Expected end of definition. [error at 92]
int kref_put_lock (struct kref *kref, void (*release)(struct kref *kref), spinlock_t *lock) __cond_acquires(true# lock)
--------------------------------------------------------------------------------------------^
The regex is suggested by Mauro; mine was too greedy. Thanks.
Updated context analysis and lock macros list provided by PeterZ. Thanks.
[mchehab: modified to be applied after xforms_lists split]
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Closes: https://lore.kernel.org/all/20260107161548.45530e1c@canb.auug.org.au/
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <3c7fdfc364a8920f92530b47bdbf4bb29a40371f.1772469446.git.mchehab+huawei@kernel.org>
Over the time, most of the changes at kernel-doc are related
to maintaining a list of transforms to convert macros into pure
C code.
Place such transforms on a separate module, to cleanup the
parser module.
There is an advantage on that: QEMU also uses our own kernel-doc,
but the xforms list there is different. By placing it on a
separate module, we can minimize the differences and make it
easier to keep QEMU in sync with Kernel upstream.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ccd74b7589e1fff340a74bf8ed16a974532cb54f.1772469446.git.mchehab+huawei@kernel.org>
Mangling with #defines is not nice, as we may end removing
the macro names, preventing several macros from being properly
documented.
Also, on defines, we have something like:
#define foo(a1, a2, a3, ...) \
/* some real implementation */
The prototype part (first line on this example) won't contain
any macros, so no need to apply any regexes on it.
With that, move the apply_transforms() logic to ensure that
it will be called only on functions.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8f9854c8ca1c794b6a3fe418f7adbc32aa68b432.1772469446.git.mchehab+huawei@kernel.org>
