Commit Graph

1427040 Commits

Author SHA1 Message Date
Manuel Ebner
ece7e57afd docs: changes.rst and ver_linux: sort the lists
Sort the lists of tools in both scripts/ver_linux and
Documentation/process/changes.rst into alphabetical order, facilitating
comparison between the two.

Signed-off-by: Manuel Ebner <manuelebner@mailbox.org>
[jc: rewrote changelog]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260325194811.78509-2-manuelebner@mailbox.org>
2026-03-30 10:36:11 -06:00
Manuel Ebner
d8a224f519 docs: changes/ver_linux: fix entries and add several tools
Some of the entries in both Documentation/process/changes.rst and
script/ver_linux were obsolete; update them to reflect the current way of
getting version information.

Many were missing altogether; add the relevant information for:

 bash, bc, bindgen, btrfs-progs, Clang, gdb,  GNU awk, GNU tar,
 GRUB, GRUB2, gtags, iptables, kmod, mcelog, mkimage, openssl,
 pahole, Python, Rust, Sphinx, squashfs-tools

Signed-off-by: Manuel Ebner <manuelebner@mailbox.org>
[jc: rewrote changelog]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260325194616.78093-2-manuelebner@mailbox.org>
2026-03-30 10:31:18 -06:00
Jonathan Corbet
06dbdc5da1 Revert "scripts: ver_linux: expand and fix list"
This reverts commit 98e7b57528.

I had not intended to apply this version of this patch; take it out and
we'll try again later.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2026-03-30 10:29:54 -06:00
Florian Fainelli
7f7f468548 Documentation: Provide hints on how to debug Python GDB scripts
By default GDB does not print a full stack of its integrated Python
interpreter, thus making the debugging of GDB scripts more painful than
it has to be.

Suggested-by: Radu Rendec <radu@rendec.net>
Signed-off-by: Florian Fainelli <florian.fainelli@broadcom.com>
Reviewed-by: Radu Rendec <radu@rendec.net>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260326233226.2248817-1-florian.fainelli@broadcom.com>
2026-03-30 10:04:13 -06:00
Mauro Carvalho Chehab
d642acfd59 doc tools: better handle KBUILD_VERBOSE
As reported by Jacob, there are troubles when KBUILD_VERBOSE is
set at the environment.

Fix it on both kernel-doc and sphinx-build-wrapper.

Reported-by: Jacob Keller <jacob.e.keller@intel.com>
Closes: https://lore.kernel.org/linux-doc/9367d899-53af-4d9c-9320-22fc4dbadca5@intel.com/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <7a99788db75630fb14828d612c0fd77c45ec1891.1774591065.git.mchehab+huawei@kernel.org>
2026-03-30 10:03:40 -06:00
Jonathan Corbet
3a61439462 Merge branch 'docs-fixes' into docs-mw
Bring the checkpatch Assisted-by fix into docs-next as well.
2026-03-30 10:01:39 -06:00
Harry Wentland
8545d9bc4b scripts/checkpatch: add Assisted-by: tag validation
The coding-assistants.rst documentation defines the Assisted-by: tag
format for AI-assisted contributions as:

  Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

This format does not use an email address, so checkpatch currently
reports a false positive about an invalid email when encountering this
tag.

Add Assisted-by: to the recognized signature tags and standard signature
list. When an Assisted-by: tag is found, validate it instead of checking
for an email address.

Examples of passing tags:
- Claude:claude-3-opus coccinelle sparse
- FOO:BAR.baz
- Copilot Github:claude-3-opus
- GitHub Copilot:Claude Opus 4.6
- My Cool Agent:v1.2.3 coccinelle sparse

Examples of tags triggering the new warning:
- Claude coccinelle sparse
- JustAName
- :missing-agent

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Assisted-by: Claude:claude-opus-4.6
Co-developed-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Alex Hung <alex.hung@amd.com>
Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260327154157.162962-1-harry.wentland@amd.com>
2026-03-30 10:00:40 -06:00
Akiyoshi Kurita
0a4f3ef988 docs: ja_JP: process: translate second half of 'Describe your changes'
Translate the remaining part of the "Describe your changes" section in
Documentation/translations/ja_JP/process/submitting-patches.rst.

Follow review comments on wording and line wrapping, and cover guidance
on self-contained patch descriptions, imperative mood, commit
references, and Link:/Closes:/Fixes: tags.

Signed-off-by: Akiyoshi Kurita <weibu@redadmin.org>
Acked-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260309105015.309116-1-weibu@redadmin.org>
2026-03-25 13:44:08 -06:00
Daniel Pereira
aa3bee451d docs: pt_BR: Add translation for KVM x86 maintainer guide
Translate the KVM x86 maintainer guidelines (maintainer-kvm-x86.rst)
into Portuguese (pt_BR). This document covers the specific
workflow, coding style, and testing requirements for the
KVM x86 subsystem.

Signed-off-by: Daniel Pereira <danielmaraboo@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260323171133.88074-3-danielmaraboo@gmail.com>
2026-03-25 13:41:54 -06:00
Daniel Pereira
1329cc0b8d docs: pt_BR: Add translation for process/conclave.rst
Translate the Linux kernel project continuity documentation (conclave.rst)
into Portuguese (pt_BR). Also, update the main pt_BR index to include
the link to the new translation.

Signed-off-by: Daniel Pereira <danielmaraboo@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260323171133.88074-2-danielmaraboo@gmail.com>
2026-03-25 13:41:54 -06:00
Mauro Carvalho Chehab
2ca0b54dca docs: c_lex.py: store logger on its data
By having the logger stored there, any code using CTokenizer can
log messages there.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <467979dc18149e4b2a7113c178e0cb07919632f2.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:46 -06:00
Mauro Carvalho Chehab
9c3911812b docs: kdoc_output: raise an error if full_proto not available for var
This is mandatory, but if it is missing, we need to know what
symbol had problems.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <0c3d9dd25889784b999efdb354ade48264c0e03c.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:46 -06:00
Mauro Carvalho Chehab
e786fab2cf docs: kdoc_yaml_file: use a better name for the tests
Instead of always using a name with a number on it, use
the name of the object directly whenever possible.

When the name is already used, append a number prefix at
the end.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <d1c4cd94547d843af0debf9e317e006d55d705f1.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:46 -06:00
Mauro Carvalho Chehab
99ec67a998 docs: kdoc: better handle source when producing YAML output
The current logic was storing symbols source code on a list,
not linked to the actual KdocItem. While this works fine when
kernel-doc markups are OK, on places where there is a "/**"
without a valid kernel-doc markup, it ends that the 1:1 match
between source code and KdocItem doesn't happen, causing
problems to generate the YAML output.

Fix it by storing the source code directly into the KdocItem
structure.

This shouldn't affect performance or memory footprint, except
when --yaml option is used.

While here, add a __repr__() function for KdocItem, as it
helps debugging it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <77902dafabb5c3250486aa2dc1568d5fafa95c5b.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:46 -06:00
Mauro Carvalho Chehab
8326e4a218 docs: kdoc_output: fix handling of simple tables
Fix check for simple table delimiters.

ReST simple tables use "=" instead of "-". I ended testing it with
a table modified from a complex one, using "--- --- ---", instead
of searching for a real Kernel example.

Only noticed when adding an unit test and seek for an actual
example from kernel-doc markups.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <dea95337c05040f95e5a95ae41d69ddef0aaa8d6.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:46 -06:00
Mauro Carvalho Chehab
bc63427c4d docs: kdoc-test.yaml: add more tests
Add extra tests to check if the new "var" type is properly
handled and to cover mutex context annotations.

Co-developed-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <75af93a45151b630c94b7f77637d173e6119fd41.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:45 -06:00
Mauro Carvalho Chehab
6e0d7b6367 docs: kdoc_yaml_file: add a representer to make strings look nicer
The strings representation is not ok, currently. Add a helper
function to improve it, and drop blank lines at beginning and
at the end of the dumps

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <422041a8b49b2609de5749092fe074b7948c32a6.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:45 -06:00
Mauro Carvalho Chehab
e896174e46 docs: tools: include kdoc_yaml_file at documentation
Add an autodoc entry for the new kdoc_yaml_file module.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <32b86abe7acee2dd4f73a35836ec94e8690f04cd.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:45 -06:00
Mauro Carvalho Chehab
3f049a5ba9 unittests: test_kdoc_parser: add command line arg to read a YAML file
The test_kdoc_parser.py already supports loading dynamic tests
when running unit tests.

Add support to read from a different file. This is useful for:
- regression tests before/afer some changes;
- preparing new unit tests;
- test a different yaml before adding its contents at
  tools/unittests/kdoc-test.yaml.

It should be noticed that passing an argument to a unit test
is not too trivial, as unittest core will load itself the
runner with a separate environment. The best (only?) way to
do it is by setting the system environment. This way, when
the class is called by the unit test loader, it can pick
the var from the environment without relying on a global
variable.

The unittest_helper has already provision for it, so let's
use its support.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <1d1a56de012c43756f9ca87aa9bf6c285674f113.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:45 -06:00
Mauro Carvalho Chehab
625ce45f2c MAINTAINERS: update documentation scripts to add unittests
Ensure that we'll receive e-mails for attempts to touch
tools/unittests.

While here, place entries alphabetically sorted.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <6be396b8423372dd006484351ca2dbc54220587e.1774256269.git.mchehab+huawei@kernel.org>
2026-03-25 13:36:45 -06:00
Rito Rhymes
c464407ada docs: allow long links to wrap per character to prevent page overflow
Some documentation pages contain long link text without natural
break points, which can force page-wide horizontal scroll overflow
on small screens.

Use overflow-wrap: anywhere for anchor text in the docs stylesheet so
links can wrap per character as a fallback when normal word boundaries
are unavailable.

Examples:
  https://docs.kernel.org/6.15/firmware-guide/acpi/non-d0-probe.html
  https://docs.kernel.org/6.15/arch/x86/earlyprintk.html

Signed-off-by: Rito Rhymes <rito@ritovision.com>
Assisted-by: Codex:GPT-5.4
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260323152428.30483-1-rito@ritovision.com>
2026-03-25 13:22:02 -06:00
Rito Rhymes
127bdd9f12 docs: allow long table reference links to wrap and prevent overflow
Some documentation pages contain docutils tables with reference links
that use long unbroken strings. Those strings can expand the table
width beyond the content column and cause page-wide horizontal
overflow.

Allow reference links in docutils tables in the main document body to
wrap when needed so the table stays within the content column and does
not break page layout.

Examples:
  https://docs.kernel.org/6.15/arch/openrisc/openrisc_port.html
  https://docs.kernel.org/6.15/filesystems/ext2.html

Signed-off-by: Rito Rhymes <rito@ritovision.com>
Assisted-by: Codex:GPT-5.4
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260323152428.30483-2-rito@ritovision.com>
2026-03-25 13:18:39 -06:00
Rito Rhymes
3aff388bc4 docs: contain horizontal overflow in C API descriptions
Some documentation pages contain long C API signatures that can exceed
the content width and cause page-wide horizontal scroll overflow.

Apply contained horizontal scrolling to C API description blocks and
keep their signature rows on one line. This preserves signature
formatting while preventing them from breaking page layout.

Contained horizontal scrolling is preferred over wrapping here because
code fidelity is the priority. These blocks are intended to remain
representative of the code itself. Wrapping distorts spacing and line
structure, which affects fidelity, creates misleading renderings, and
reduces readability.

Examples:
  https://docs.kernel.org/6.15/driver-api/regulator.html
  https://docs.kernel.org/6.15/userspace-api/fwctl/fwctl-cxl.html

Signed-off-by: Rito Rhymes <rito@ritovision.com>
Assisted-by: Codex:GPT-5.4
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260323153342.33447-1-rito@ritovision.com>
2026-03-25 13:17:13 -06:00
Rito Rhymes
f2679ac773 docs: allow inline literals in paragraphs to wrap to prevent overflow
Some documentation pages contain long inline literals in paragraph
text that can force page-wide horizontal scroll overflow and break
layout on smaller screens.

Override the default `span.pre` white-space behavior for inline
literals and use `overflow-wrap: anywhere` so they can wrap when
needed. For code used as part of a paragraph, wrapping is appropriate
because it is stylistically part of the surrounding text. Code blocks,
by contrast, are meant to preserve formatting fidelity and are better
served by contained horizontal scrolling.

Examples:
  https://docs.kernel.org/6.15/userspace-api/futex2.html
  https://docs.kernel.org/6.15/security/IMA-templates.html

Signed-off-by: Rito Rhymes <rito@ritovision.com>
Assisted-by: Codex:GPT-5.4
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260323151401.27415-1-rito@ritovision.com>
2026-03-25 13:16:58 -06:00
Tomás Pando
19dcccbc06 docs: driver-api: fix 6 spelling typos in Documentation/driver-api
Fix minor spelling mistakes in the driver-api documentation. These
changes improve readability in ACPI, CXL, DMA and PCI docs.

Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Tomás Pando <tovictakamine@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260324163604.5710-1-tovictakamine@gmail.com>
2026-03-25 12:36:02 -06:00
Rito Rhymes
99aa500409 docs: use logo.svg as favicon
Use the existing documentation logo as the HTML favicon.

This makes generated documentation pages use a matching browser tab
icon without introducing a separate favicon asset.

Signed-off-by: Rito Rhymes <rito@ritovision.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260321125532.9568-1-rito@ritovision.com>
2026-03-22 15:27:43 -06:00
Jonathan Corbet
0a1a27776d Merge branch 'mauro' into docs-mw
This series comes after:
    https://lore.kernel.org/linux-doc/cover.1773770483.git.mchehab+huawei@kernel.org/

It basically contains patches I submitted before on a 40+ patch series,
but were less relevant, plus a couple of other minor fixes:

- patch 1 improves one of the CTokenizer unit test, fixing some
  potential issues on it;
- patches 2 and 3 contain some improvement/fixes for Sphinx
  Python autodoc extension. They basically document c_lex.py;

- The remaining patches:
  - create a new class for kernel-doc config;
  - fix some internal representations of KdocItem;
  - add unit tests for KernelDoc() parser class;
  - add support to output KdocItem in YAML, which is a
    machine-readable output for all documented kAPI.

None of the patches should affect man or html output.
2026-03-22 15:25:08 -06:00
Mauro Carvalho Chehab
01d6d7bf96 docs: kernel-doc: add support to store output on a YAML file
Add a command line parameter and library support to optionally
store:
- KdocItem intermediate format after parsing;
- man pages output;
- rst output.

inside a YAML file.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ba54277b3c909867153b9547dfa33c1831ca35d9.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
b37b3cbbb1 docs: add a new file to write kernel-doc output to a YAML file
Storing kernel-doc output is helpful to allow debugging problems
on it and to preparate unit tests.

Add a class to store such contents at the same format as defined
at kdoc-test-schema.yaml.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <5d084ca1a91f6a620534a1135d1b8183d934319a.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
eea0d807db docs: test_kdoc_parser: add support for dynamic test creation
Use the content of kdoc-test.yaml to generate unittests to
verify that kernel-doc internal methods are parsing C code
and generating output the expected way.

Depending on what is written at the parser file at
kdoc-test.yaml, up to 5 tests can be generated from a single
test entry inside the YAML file:

1. from source to kdoc_item: test KernelDoc class;
2. from kdoc_item to man: test ManOutput class;
3. from kdoc_item to rst: test RestOutput class;
4. from source to man without checking expected KdocItem;
5. from source to rst without checking expected KdocItem.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <7ec2593c5b19de3e3b1d8de92675f6b751d3fa21.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
8b69f5225a docs: add a simple kdoc-test.yaml together with a validation tool
Create a simple kdoc-test.yaml to be used to create unit tests for
kernel-doc parser and output classes.

For now, all we want is a simple function mapped on a yaml test
using the defined schema.

To be sure that the schema is followed, add an unittest for
the file, which will also validate that the schema is properly
parsed.

It should be noticed that the .TH definition for the man format
contains a timestamp. We'll need to handle that when dealing with
the actual implementation for the ManOutput class unit tests.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <74883976348c964f00161696d525c33ddd8c7641.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
f1cdbd824c docs: add a schema to help creating unittests for kernel-doc
Instead of hardcoding lots of tests inside a file, let's place
them inside a yaml file.

Add first a schema to handle it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <f42fd816ee0d257c736b30977b10f04f6bea27c2.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
a5dea56ebd docs: unittests: add a parser to test kernel-doc parser logic
Validating that kernel-doc is parsing data properly is tricky.

Add an unittest skeleton that alllows passing a source code
and check if the corresponding values of export_table and
entries returned by the parser are properly filled.

It works by mocking a file input with the contents of a source
string, an comparing if:

	- exports set matches;
	- expected KernelItem entries match.

Create a new TestSelfValidate meant to check if the logic
inside KdocParser.run_test() does its job of checking for
differences inside KdocItem.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8d91bfabd69de7aa44a0f5080ccb01aa41957e6d.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
e394855fcc docs: kdoc_item: fix a typo on sections_start_lines
Currently, there are 15 occurrences of section?_start_lines,
with 10 using the plural way.

This is an issue, as, while kdoc_output works with KdocItem,
the term doesn't match its init value.

The variable sections_start_lines stores multiple sections,
so placing it in plural is its correct way.

So, ensure that, on all parts of kdoc, this will be referred
as sections_start_lines.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <d1e0f1d3f80df41c11a1bbde6a12fd9468bc3813.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
99364ba7f8 docs: kdoc_item: add support to generate a KdocItem from a dict
When reading the contents on a KdocItem using YAML, the data
will be imported into a dict.

Add a method to create a new KdocItem from a dict to allow
converting such input into a real KdocItem.

While here, address an issue that, if the class is initialized
with an internal parameter outside the 4 initial arguments,
it would end being added inside other_stuff, which breaks
initializing it from a dict.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <fafeac23d1577927e1a3c32cddfbec1e0209ac73.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
01c41b99c6 docs: kdoc_item: fix initial value for parameterdesc_start_lines
Ensure that parameterdesc_start_lines is a dict at init time,
as this is how it will be set later on at the parser.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <1b4ea24dd4cd82e6711e9be80168684427d74c30.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
9ab2ca3dd1 docs: kdoc_files: move output symbols logic to kdoc_output
When writing unittests for kdoc_output, it became clear that
the logic with handles a series of KdocItem symbols from
a single file belons to kdoc_output, and not to kdoc_files.

Move the code to it.

While here, also ensure that self.config will be placed
together with set.out_style.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <4ebc26e37a0b544c50d50b8077760f147fa6a535.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
e0ebee442d docs: kdoc_files: use a class to group config parameters
Instead of abusing argparse.Namespace, define a class to store
configuration parameters and logger.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <a66ec9872c72a3ba1a5ac567881d67dc8ee581c6.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
8c0b7c0d3c docs: kdoc: add c_lex to generated documentation
Do some fixes at groups() description for it to be parsed by
Sphinx and add it to the documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <799178cf30dd4022fdb1d029ba998a458e037b52.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
b2d231f4a7 docs: kdoc_re: better represent long regular expressions
The Sphinx output from autodoc doesn't automatically break long
lines, except on spaces.

Change KernRe __repr__() to break the pattern on multiple strings,
each one with a maximum limit of 60 characters.

With that, documentation output for KernRe should now be displayable,
even on long strings.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <60c264a9d277fed655b1a62df2195562c8596090.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:40 -06:00
Mauro Carvalho Chehab
14b7775ef7 unittests: test_tokenizer: better handle mismatch error
The current logic is too sensitive to how c_lex name is
placed. Also, it doesn't really check the log.

Change it to check if the expected message will be reported
after a call to C tokenizer with an invalid source.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <6e19578bc1ffa96e536dc31997ff658017f60173.1773823995.git.mchehab+huawei@kernel.org>
2026-03-22 15:10:39 -06:00
Jonathan Corbet
781171bec0 Merge branch 'mauro' into docs-mw
Mauro says:

This patch series change how kdoc parser handles macro replacements.

Instead of heavily relying on regular expressions that can sometimes
be very complex, it uses a C lexical tokenizer. This ensures that
BEGIN/END blocks on functions and structs are properly handled,
even when nested.

Checking before/after the patch series, for both man pages and
rst only had:
    - whitespace differences;
    - struct_group macros now are shown as inner anonimous structs
      as it should be.

Also, I didn't notice any relevant change on the documentation build
time. With that regards, right now, every time a CMatch replacement
rule takes in place, it does:

    for each transform:
    - tokenizes the source code;
    - handle CMatch;
    - convert tokens back to a string.

A possible optimization would be to do, instead:

    - tokenizes source code;
    - for each transform handle CMatch;
    - convert tokens back to a string.

For now, I opted not do do it, because:

    - too much changes on a single row;
    - docs build time is taking ~3:30 minutes, which is
      about the same time it ws taken before the changes;
    - there is a very dirty hack inside function_xforms:
	 (KernRe(r"_noprof"), ""). This is meant to change
      function prototypes instead of function arguments.

So, if ok for you, I would prefer to merge this one first. We can later
optimize kdoc_parser to avoid multiple token <-> string conversions.

-

One important aspect of this series is that it introduces unittests
for kernel-doc. I used it a lot during the development of this series,
to ensure that the changes I was doing were producing the expected
results. Tests are on two separate files that can be executed directly.

Alternatively, there is a run.py script that runs all of them (and
any other python script named  tools/unittests/test_*.py"):

  $ tools/unittests/run.py
  test_cmatch:
      TestSearch:
	  test_search_acquires_multiple:                               OK
	  test_search_acquires_nested_paren:                           OK
	  test_search_acquires_simple:                                 OK
	  test_search_must_hold:                                       OK
	  test_search_must_hold_shared:                                OK
	  test_search_no_false_positive:                               OK
	  test_search_no_function:                                     OK
	  test_search_no_macro_remains:                                OK
      TestSubMultipleMacros:
	  test_acquires_multiple:                                      OK
	  test_acquires_nested_paren:                                  OK
	  test_acquires_simple:                                        OK
	  test_mixed_macros:                                           OK
	  test_must_hold:                                              OK
	  test_must_hold_shared:                                       OK
	  test_no_false_positive:                                      OK
	  test_no_function:                                            OK
	  test_no_macro_remains:                                       OK
      TestSubSimple:
	  test_rise_early_greedy:                                      OK
	  test_rise_multiple_greedy:                                   OK
	  test_strip_multiple_acquires:                                OK
	  test_sub_count_parameter:                                    OK
	  test_sub_mixed_placeholders:                                 OK
	  test_sub_multiple_placeholders:                              OK
	  test_sub_no_placeholder:                                     OK
	  test_sub_single_placeholder:                                 OK
	  test_sub_with_capture:                                       OK
	  test_sub_zero_placeholder:                                   OK
      TestSubWithLocalXforms:
	  test_functions_with_acquires_and_releases:                   OK
	  test_raw_struct_group:                                       OK
	  test_raw_struct_group_tagged:                                OK
	  test_struct_group:                                           OK
	  test_struct_group_attr:                                      OK
	  test_struct_group_tagged_with_private:                       OK
	  test_struct_kcov:                                            OK
	  test_vars_stackdepot:                                        OK

  test_tokenizer:
      TestPublicPrivate:
	  test_balanced_inner_private:                                 OK
	  test_balanced_non_greddy_private:                            OK
	  test_balanced_private:                                       OK
	  test_no private:                                             OK
	  test_unbalanced_inner_private:                               OK
	  test_unbalanced_private:                                     OK
	  test_unbalanced_struct_group_tagged_with_private:            OK
	  test_unbalanced_two_struct_group_tagged_first_with_private:  OK
	  test_unbalanced_without_end_of_line:                         OK
      TestTokenizer:
	  test_basic_tokens:                                           OK
	  test_depth_counters:                                         OK
	  test_mismatch_error:                                         OK

  Ran 47 tests
2026-03-22 15:06:59 -06:00
Mauro Carvalho Chehab
7538df7a2d docs: xforms_lists: use CMatch for all identifiers
CMatch is lexically correct and replaces only identifiers,
which is exactly where macro transformations happen.

Use it to make the output safer and ensure that all arguments
will be parsed the right way, even on complex cases.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <86d4a07ff0e054207747fabf38d6bb261b52b5fa.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
79d881beb7 docs: kdoc_parser: avoid tokenizing structs everytime
Most of the rules inside CTransforms are of the type CMatch.

Don't re-parse the source code every time.

Doing this doesn't change the output, but makes kdoc almost
as fast as before the tokenizer patches:

    # Before tokenizer patches
    $ time ./scripts/kernel-doc . -man >original 2>&1

    real    0m42.933s
    user    0m36.523s
    sys     0m1.145s

    # After tokenizer patches
    $ time ./scripts/kernel-doc . -man >before 2>&1

    real    1m29.853s
    user    1m23.974s
    sys     0m1.237s

    # After this patch
    $ time ./scripts/kernel-doc . -man >after 2>&1

    real    0m48.579s
    user    0m45.938s
    sys     0m0.988s

    $ diff -s before after
    Files before and after are identical

Manually checked the differences between original and after
with:

    $ diff -U0 -prBw original after|grep -v Warning|grep -v "@@"|less

They're due:
  - whitespace fixes;
  - struct_group are now better handled;
  - several badly-generated man pages from broken inline kernel-doc
    markups are now fixed.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <1cc2a4286ebf7d4b2d03fcaf42a1ba9fa09004b9.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
12aa7753ff docs: kdoc: ensure that comments are dropped before calling split_struct_proto()
Changeset 2b957decdb6c ("docs: kdoc: don't add broken comments inside prototypes")
revealed a hidden bug at split_struct_proto(): some comments there may break
its capability of properly identifying a struct.

Fixing it is as simple as stripping comments before calling it.

Fixes: 2b957decdb6c ("docs: kdoc: don't add broken comments inside prototypes")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <dcff37b6da5329aea415de31f543b6a1c2cbbbce.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
827b9458c9 docs: kernel-doc.rst: document private: scope propagation
This was an undefined behavior, but at least one place used private:
inside a nested struct meant to not be propagated outside it.

Kernel-doc now defines how this is propagated. So, document that.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <bbe0ed698c21f3f930a561b885bc8a47824f7f1d.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
024e200e2a docs: c_lex: setup a logger to report tokenizer issues
Report file that has issues detected via CMatch and CTokenizer.

This is done by setting up a logger that will be overriden by
kdoc_parser, when used on it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <903ad83ae176196a50444e66177a4f5bcdef5199.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
2f07ddbd57 docs: xforms_lists: better evaluate struct_group macros
The previous approach were to unwind nested structs/unions.

Now that we have a logic that can handle it well, use it to
ensure that struct_group macros will properly reflect the
actual struct.

Note that the replacemend logic still simplifies the code
a little bit, as the basic build block for struct group is:

	union { \
		struct { MEMBERS } ATTRS; \
		struct __struct_group_tag(TAG) { MEMBERS } ATTRS NAME; \
	} ATTRS

There:

- ATTRS is meant to add extra macro attributes like __packed
  which we already discard, as they aren't relevant to
  document struct members;

- TAG is used only when built with __cplusplus.

So, instead, convert them into just:

    struct { MEMBERS };

Please notice that here, we're using the greedy version of the
backrefs, as MEMBERS is actually MEMBERS... on all such macros.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <24bf2c036b08814d9b4aabc27542fd3b2ff54424.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
f63e6163c7 docs: xforms_lists: handle struct_group directly
The previous logic was handling struct_group on two steps.
Remove the previous approach, as CMatch can do it the right
way on a single step.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <da7f879d90e3ffbc1f47771522f212a60df1fab6.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00
Mauro Carvalho Chehab
ae63a5b920 docs: kdoc_re: get rid of NestedMatch class
Now that everything was converted to CMatch, we can get rid of
the previous NestedMatch implementation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <c82dd0d2c0ab330fc04925965091c448ccabb8fd.1773770483.git.mchehab+huawei@kernel.org>
2026-03-22 15:02:29 -06:00