Commit Graph

123 Commits

Author SHA1 Message Date
Nícolas F. R. A. Prado
d82b1e833e docs: Add automatic cross-reference for C types
In order to cross-reference C types in the documentation, Sphinx
requires the syntax :c:type:`type_name`, or even :c:type:`struct
type_name <type_name>` in order to have the link text different from the
target text.

Extend automarkup to enable automatic cross-reference of C types by
matching any "struct|union|enum|typedef type_name" expression.
This makes the documentation's plain text cleaner and adds
cross-reference to types without any additional effort by the author.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20200903005747.3900333-2-nfraprado@protonmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-09-03 13:50:40 -06:00
Alexander A. Klimov
a5989e29ac Replace HTTP links with HTTPS ones: Documentation/sphinx/parse-headers.pl
Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.

Deterministic algorithm:
For each file:
  For each line:
    If doesn't contain `\bxmlns\b`:
      For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
        If both the HTTP and HTTPS versions
        return 200 OK and serve the same content:
          Replace HTTP with HTTPS.

Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Link: https://lore.kernel.org/r/20200620075402.22347-1-grandmaster@al2klimov.de
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-06-26 10:57:33 -06:00
Alexander A. Klimov
93431e0607 Replace HTTP links with HTTPS ones: documentation
Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.

Deterministic algorithm:
For each file:
  For each line:
    If doesn't contain `\bxmlns\b`:
      For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
        If both the HTTP and HTTPS versions
        return 200 OK and serve the same content:
          Replace HTTP with HTTPS.

Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Link: https://lore.kernel.org/r/20200526060544.25127-1-grandmaster@al2klimov.de
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-06-08 09:30:19 -06:00
Mauro Carvalho Chehab
d5afc9640a docs: update recommended Sphinx version to 2.4.4
There are some docs that have nested tables. While this was
always part of the spec, only Sphinx version 2.4.x can
translate it to LaTeX.

In other words, if someone is using a Sphinx version < 2.4,
the LaTeX and PDF output won't work for some of the docs.

So, it seems that it is time to raise the bar again
for the recommented version.

The Sphinx check script is already smart enough to keep
working, with older versions, warning the users that
an upgrade is recommended (and explaining how):

	Sphinx version 1.7.9
	Warning: It is recommended at least Sphinx version 2.4.4.
	Detected OS: Fedora release 31 (Thirty One).

	To upgrade Sphinx, use:

		/usr/bin/virtualenv sphinx_2.4.4
		. sphinx_2.4.4/bin/activate
		pip install -r ./Documentation/sphinx/requirements.txt

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/498f701c618f7d0cf5f0a37e5889ee926f7c8bf4.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-04-20 15:35:58 -06:00
Kees Cook
adc10f5b0a docs: Fix empty parallelism argument
When there was no parallelism (no top-level -j arg and a pre-1.7
sphinx-build), the argument passed would be empty ("") instead of just
being missing, which would (understandably) badly confuse sphinx-build.
Fix this by removing the quotes.

Reported-by: Rafael J. Wysocki <rafael@kernel.org>
Fixes: 51e46c7a40 ("docs, parallelism: Rearrange how jobserver reservations are made")
Cc: stable@vger.kernel.org  # v5.5 only
Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-02-25 03:11:04 -07:00
Jonathan Corbet
bcac386f3d docs: Keep up with the location of NoUri
Sphinx 2.1 moved sphinx.environment.NoUri into sphinx.errors; that produced
this warning in the docs build:

  /usr/lib/python3.7/site-packages/sphinx/registry.py:473:
    RemovedInSphinx30Warning: sphinx.environment.NoUri is deprecated.

Grab NoUri from the right place and make the warning go away.  That symbol
was only added to sphinx.errors in 2.1, so we must still import it from the
old location when running in older versions.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-01-24 09:47:05 -07:00
Kees Cook
51e46c7a40 docs, parallelism: Rearrange how jobserver reservations are made
Rasmus correctly observed that the existing jobserver reservation only
worked if no other build targets were specified. The correct approach
is to hold the jobserver slots until sphinx has finished. To fix this,
the following changes are made:

- refactor (and rename) scripts/jobserver-exec to set an environment
  variable for the maximally reserved jobserver slots and exec a
  child, to release the slots on exit.

- create Documentation/scripts/parallel-wrapper.sh which examines both
  $PARALLELISM and the detected "-jauto" logic from Documentation/Makefile
  to decide sphinx's final -j argument.

- chain these together in Documentation/Makefile

Suggested-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Link: https://lore.kernel.org/lkml/eb25959a-9ec4-3530-2031-d9d716b40b20@rasmusvillemoes.dk
Signed-off-by: Kees Cook <keescook@chromium.org>
Link: https://lore.kernel.org/r/20191121205929.40371-4-keescook@chromium.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-11-22 10:35:18 -07:00
Changbin Du
36bc683dde kernel-doc: rename the kernel-doc directive 'functions' to 'identifiers'
The 'functions' directive is not only for functions, but also works for
structs/unions. So the name is misleading. This patch renames it to
'identifiers', which specific the functions/types to be included in
documentation. We keep the old name as an alias of the new one before
all documentation are updated.

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-11-07 13:17:25 -07:00
Kees Cook
aa20485528 doc-rst: Programmatically render MAINTAINERS into ReST
In order to have the MAINTAINERS file visible in the rendered ReST
output, this makes some small changes to the existing MAINTAINERS file
to allow for better machine processing, and adds a new Sphinx directive
"maintainers-include" to perform the rendering.

Features include:
- Per-subsystem reference links: subsystem maintainer entries can be
  trivially linked to both internally and external. For example:
  https://www.kernel.org/doc/html/latest/process/maintainers.html#secure-computing

- Internally referenced .rst files are linked so they can be followed
  when browsing the resulting rendering. This allows, for example, the
  future addition of maintainer profiles to be automatically linked.

- Field name expansion: instead of the short fields (e.g. "M", "F",
  "K"), use the indicated inline "full names" for the fields (which are
  marked with "*"s in MAINTAINERS) so that a rendered subsystem entry
  is more human readable. Email lists are additionally comma-separated.
  For example:

    SECURE COMPUTING
	Mail:	  Kees Cook <keescook@chromium.org>
	Reviewer: Andy Lutomirski <luto@amacapital.net>,
		  Will Drewry <wad@chromium.org>
	SCM:	  git git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git seccomp
	Status:	  Supported
	Files:	  kernel/seccomp.c include/uapi/linux/seccomp.h
		  include/linux/seccomp.h tools/testing/selftests/seccomp/*
		  tools/testing/selftests/kselftest_harness.h
		  userspace-api/seccomp_filter
	Content regex:	\bsecure_computing \bTIF_SECCOMP\b

Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-10-02 10:03:17 -06:00
Jonathan Neuschäfer
82bf829b69 Documentation: sphinx: Don't parse socket() as identifier reference
With the introduction of Documentation/sphinx/automarkup.py, socket() is
parsed as a reference to the in-kernel definition of socket. Sphinx then
decides that struct socket is a good match, which is usually not
intended, when the syscall is meant instead. This was observed in
Documentation/networking/af_xdp.rst.

Prevent socket() from being misinterpreted by adding it to the Skipfuncs
list in automarkup.py.

Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Reviewed-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-08-12 14:55:30 -06:00
Jonathan Neuschäfer
11fec009d9 Documentation: sphinx: Add missing comma to list of strings
In Python, like in C, when a comma is omitted in a list of strings, the
two strings around the missing comma are concatenated.

Cc: stable@vger.kernel.org  # v5.2 only
Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Reviewed-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-08-12 14:54:22 -06:00
Mauro Carvalho Chehab
d6f0f2f19a docs: load_config.py: ensure subdirs end with "/"
The logic with seeks for a subdir passed via SPHINXDIRS is
incomplete: if one uses something like:

	make SPHINXDIRS=arm pdfdocs

It will find both "arm" and "arm64" directories. Worse than
that, it will convert "arm64/index" to "4/index".

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
2019-07-19 08:49:27 -03:00
Mauro Carvalho Chehab
a84d9e8996 docs: load_config.py: avoid needing a conf.py just due to LaTeX docs
Right now, for every directory that we need to have LaTeX output,
a conf.py file is required.

That causes an extra overhead and it is actually a hack, as
the latex_documents line there are usually a copy of the ones
that are there already at the main conf.py.

So, instead, re-use the global latex_documents var, just
adjusting the path to be relative ones.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
2019-07-17 06:57:52 -03:00
Mauro Carvalho Chehab
454f96f2b7 docs: automarkup.py: ignore exceptions when seeking for xrefs
When using the automarkup extension with:
	make pdfdocs

without passing an specific book, the code will raise an exception:

	  File "/devel/v4l/docs/Documentation/sphinx/automarkup.py", line 86, in auto_markup
	    node.parent.replace(node, markup_funcs(name, app, node))
	  File "/devel/v4l/docs/Documentation/sphinx/automarkup.py", line 59, in markup_funcs
	    'function', target, pxref, lit_text)
	  File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/domains/c.py", line 308, in resolve_xref
	    contnode, target)
	  File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/util/nodes.py", line 450, in make_refnode
	    '#' + targetid)
	  File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/builders/latex/__init__.py", line 159, in get_relative_uri
	    return self.get_target_uri(to, typ)
	  File "/devel/v4l/docs/sphinx_2.0/lib/python3.7/site-packages/sphinx/builders/latex/__init__.py", line 152, in get_target_uri
	    raise NoUri
	sphinx.environment.NoUri

This happens because not all references will belong to a single
PDF/LaTeX document.

Better to just ignore those than breaking Sphinx build.

Fixes: d74b0d31dd ("Docs: An initial automarkup extension for sphinx")
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
[jc: Narrowed the "except" and tweaked the comment]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-07-08 14:35:47 -06:00
Jonathan Corbet
d74b0d31dd Docs: An initial automarkup extension for sphinx
Rather than fill our text files with :c:func:`function()` syntax, just do
the markup via a hook into the sphinx build process.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-06-26 11:14:09 -06:00
Mauro Carvalho Chehab
a700767a76 docs: requirements.txt: recommend Sphinx 1.7.9
As discussed at the linux-doc ML, while we'll still support
version 1.3, it is time to recommend a more modern version.

So, let's switch the minimal requirements to Sphinx 1.7.9,
as it has the "-jauto" flag, with makes a lot faster when
building documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-05-30 10:42:05 -06:00
Mauro Carvalho Chehab
42f6ebd827 docs: cdomain.py: get rid of a warning since version 1.8
There's a new warning about a deprecation function. Add a
logic at cdomain.py to avoid that.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-05-29 15:47:32 -06:00
Jonathan Corbet
2404dad1f6 doc: Cope with the deprecation of AutoReporter
AutoReporter is going away; recent versions of sphinx emit a warning like:

  Documentation/sphinx/kerneldoc.py:125:
      RemovedInSphinx20Warning: AutodocReporter is now deprecated.
      Use sphinx.util.docutils.switch_source_input() instead.

Make the switch.  But switch_source_input() only showed up in 1.7, so we
have to do ugly version checks to keep things working in older versions.

Cc: stable@vger.kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-05-23 09:23:11 -06:00
Jonathan Corbet
096ea522e8 doc: Cope with Sphinx logging deprecations
Recent versions of sphinx will emit messages like:

  Documentation/sphinx/kerneldoc.py:103:
     RemovedInSphinx20Warning: app.warning() is now deprecated.
     Use sphinx.util.logging instead.

Switch to sphinx.util.logging to make this unsightly message go away.
Alas, that interface was only added in version 1.6, so we have to add a
version check to keep things working with older sphinxes.

Cc: stable@vger.kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-05-23 09:22:00 -06:00
Mike Rapoport
f2e8603604 Documentation/sphinx: allow "functions" with no parameters
When kernel-doc:: specified in .rst document without explicit directives,
it outputs both comment and DOC: sections. If a DOC: section was explicitly
included in the same document it will be duplicated. For example, the
output generated for Documentation/core-api/idr.rst [1] has "IDA
description" in the "IDA usage" section and in the middle of the API
reference.

This patch enables using "functions" directive without parameters to output
all the documentation excluding DOC: sections.

[1] https://www.kernel.org/doc/html/v4.17/core-api/idr.html

Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Acked-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-06-30 07:52:42 -06:00
Federico Vaga
f3821276f6 doc:sphinx: fix parse-header description
The description speaks about the option ``--man`` but it
does not exist. Instead, there is the option ``--usage``

    $ ./Documentation/sphinx/parse-headers.pl --man
    Unknown option: man
    Usage:
        parse_headers.pl [<options>] <C_FILE> <OUT_FILE> [<EXCEPTIONS_FILE>]

        Where <options> can be: --debug, --help or --man.

Signed-off-by: Federico Vaga <federico.vaga@vaga.pv.it>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-06-26 09:07:57 -06:00
Mauro Carvalho Chehab
315e6bc538 Documentation: rstFlatTable.py: fix a broken reference
The old HOWTO was removed a long time ago. The flat table
version is not metioned elsewhere, so just get rid of the
text.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
Acked-by: Jonathan Corbet <corbet@lwn.net>
2018-06-15 18:10:01 -03:00
Mauro Carvalho Chehab
3259081991 MAINTAINERS & files: Canonize the e-mails I use at files
From now on, I'll start using my @kernel.org as my development e-mail.

As such, let's remove the entries that point to the old
mchehab@s-opensource.com at MAINTAINERS file.

For the files written with a copyright with mchehab@s-opensource,
let's keep Samsung on their names, using mchehab+samsung@kernel.org,
in order to keep pointing to my employer, with sponsors the work.

For the files written before I join Samsung (on July, 4 2013),
let's just use mchehab@kernel.org.

For bug reports, we can simply point to just kernel.org, as
this will reach my mchehab+samsung inbox anyway.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Brian Warner <brian.warner@samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
2018-05-04 06:21:06 -04:00
Matthew Wilcox
ff690eeed8 Documentation/sphinx: Fix Directive import error
Sphinx 1.7 removed sphinx.util.compat.Directive so people
who have upgraded cannot build the documentation.  Switch to
docutils.parsers.rst.Directive which has been available since
docutils 0.5 released in 2009.

Bugzilla: https://bugzilla.opensuse.org/show_bug.cgi?id=1083694
Co-developed-by: Takashi Iwai <tiwai@suse.de>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-03-09 10:46:14 -07:00
Masanari Iida
ae17a87dd7 linux-next: docs-rst: Fix typos in kfigure.py
This patch fixes some spelling typos found in kfigure.py

Signed-off-by: Masanari Iida <standby24x7@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2018-01-17 16:49:05 -07:00
Jani Nikula
86c0f046a8 Documentation/sphinx: fix kernel-doc decode for non-utf-8 locale
On python3, Popen() universal_newlines=True converts the subprocess
stdout to unicode text using a codec based on user preferences. Given
LANG indicating ascii and utf-8 stdout from the subprocess, you'd get:

WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno
../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't
decode byte 0xe2 in position 6368: ordinal not in range(128)

Fix this by dropping universal_newlines=True and replacing the implicit
LANG specific decode with an explicit utf-8 decode. This also gets rid
of the annoying conditional code for python 2 vs. 3.

Fixes: ba35018593 ("Documentation/sphinx: fix kernel-doc extension on python3")
Reference: http://mid.mail-archive.com/54c23e8e-89c0-5cea-0dcc-e938952c5642@infradead.org
Reported-and-tested-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-08-31 13:36:28 -06:00
Mauro Carvalho Chehab
fb947f3f47 sphinx-pre-install: use a requirements file
Instead of using 3 commands to install a virtualenv, use
a single one, reading the requirements from this file:

	Documentation/sphinx/requirements.txt

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-07-23 15:51:24 -06:00
Jonathan Corbet
52b3f239bb Docs: clean up some DocBook loose ends
There were a few bits and pieces left over from the now-disused DocBook
toolchain; git rid of them.

Reported-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-06-23 14:17:38 -06:00
Mauro Carvalho Chehab
fd74809e73 docs-rst: get rid of Documentation/sphinx/tmplcvt script
As everything was converted to ReST, we don't need this
script anymore.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2017-05-16 08:44:24 -03:00
Mauro Carvalho Chehab
686b0d9bb4 tmplcvt: make the tool more robust
Currently, the script just assumes to be called at
Documentation/sphinx/. Change it to work on any directory,
and make it abort if something gets wrong.

Also, be sure that both parameters are specified.

That should avoid troubles like this:

$ Documentation/sphinx/tmplcvt Documentation/DocBook/writing_usb_driver.tmpl
sed: couldn't open file convert_template.sed: No such file or directory

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-04-11 14:35:15 -06:00
Rémy Léone
c46988aef2 Use sphinx.version_info directly instead of parsing
Using the development version of sphinx caused the parsing of the
version to fail.

Signed-off-by: Rémy Léone <remy.leone@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-31 09:19:37 -06:00
Markus Heiser
db6ccf23e8 docs-rst: automatically convert Graphviz and SVG images
This patch brings scalable figure, image handling and a concept to
embed *render* markups:

* DOT (http://www.graphviz.org)
* SVG

For image handling use the 'image' replacement::

    .. kernel-image::  svg_image.svg
       :alt:    simple SVG image

For figure handling use the 'figure' replacement::

    .. kernel-figure::  svg_image.svg
       :alt:    simple SVG image

       SVG image example

Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::

  .. kernel-render:: DOT
     :alt: foobar digraph
     :caption: Embedded **DOT** (Graphviz) code.

     digraph foo {
      "bar" -> "baz";
     }

The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:

* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)

Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Tested-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Tested-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> (v2 - v5)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> (v1, v6)
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2017-03-09 02:59:26 -07:00
Dmitry Shachnev
217e2bfab2 docs: sphinx-extensions: make rstFlatTable work with docutils 0.13
In docutils 0.13, the return type of get_column_widths method of the
Table directive has changed [1], which breaks our flat-table directive
and leads to a TypeError when trying to build the docs [2].

This patch adds support for the new return type, while keeping support
for older docutils versions too.

[1] https://sourceforge.net/p/docutils/patches/120/
[2] https://sourceforge.net/p/docutils/bugs/303/

Signed-off-by: Dmitry Shachnev <mitya57@debian.org>
Cc: <stable@vger.kernel.org> # 4.8.x-
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-12-18 13:30:29 -07:00
Mauro Carvalho Chehab
c339665666 docs-rst: parse-headers.pl: cleanup the documentation
Keeping both rst and in-file documentation in sync can be harsh.

So, simplify the script's internal documntation to a bare minimum,
and add a mention to the ReST file with its full documentation.

This way, a quick help is still available at the command line,
while the complete one is maintained at the ReST format.

As we won't be using pad2rst anymore, do a cleanup at the ReST
file.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-11-30 17:08:09 -07:00
Mauro Carvalho Chehab
327f5a754a parse-headers.pl: add documentation for this script
Provide a man page for parse-headers.pl, describing
how to use it.

The documentation on ReST format was generated via pod2rst:
	http://search.cpan.org/~dowens/Pod-POM-View-Restructured-0.02/bin/pod2rst

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-11-19 10:22:08 -07:00
Jani Nikula
58af04dff7 Documentation/sphinx: rename kernel-doc.py to kerneldoc.py
Python module names should not have hyphens per [PEP 8]. Drop the hyphen
from kernel-doc.py. The extension directive remains unchanged.

[PEP 8] https://www.python.org/dev/peps/pep-0008/#package-and-module-names

Reported-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-10-19 16:02:00 -06:00
Mauro Carvalho Chehab
142a0e11b5 Merge tag 'docs-next' of git://git.lwn.net/linux.git into patchwork
Merge back from docs-next in order to get the cdomain extension.

With such extension, the number of warnings when building docs
in nitpick mode reduced from 22 to 2 warnings.

* docs-next/docs-next:
  docs/driver-model: fix typo
  DMA-API-HOWTO: <asm/generic/scatterlist.h> is no more
  doc-rst:c-domain: function-like macros arguments
  doc-rst:c-domain: fix sphinx version incompatibility
  Documentation/filesystems: Fixed typo
  docs: Don't format internal MPT docs
  docs: split up serial-interfaces.rst
  docs: Pull the HSI documentation together
  docs: Special-case function-pointer parameters in kernel-doc
  docs: make kernel-doc handle varargs properly
  x86: fix memory ranges in mm documentation
  documentation/scsi: Remove nodisconnect parameter
  doc: ioctl: Add some clarifications to botching-up-ioctls
  docs: split up the driver book
  Docs: sphinxify device-drivers.tmpl
2016-09-19 16:36:41 -03:00
Markus Heiser
56cd869288 doc-rst:c-domain: function-like macros arguments
Handle signatures of function-like macros well. Don't try to deduce
arguments types of function-like macros.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-16 09:53:40 -06:00
Markus Heiser
b495360e30 doc-rst:c-domain: fix sphinx version incompatibility
The self.indexnode's tuple has changed in sphinx version 1.4, from a
former 4 element tuple to a 5 element tuple.

e6a5a3a92e

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-16 09:53:23 -06:00
Mauro Carvalho Chehab
22c4003364 [media] docs-rst: parse-headers.pl: use the C domain for cross-references
Instead of keep using the normal reference, move to the C
domain ones. Using C domains everywhere will allow
cross-references between kAPI and uAPI docs.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-09 09:20:27 -03:00
Mauro Carvalho Chehab
bcec7c218d [media] docs-rst: parse-headers.pl: make debug a command line option
Add a parser for the --debug option, in order to allow
seeing what the parser is doing.

Acked-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-09-09 09:20:06 -03:00
Markus Heiser
b62b9d81a0 docs: sphinx-extensions: add metadata parallel-safe
The setup() function of a Sphinx-extension can return a dictionary. This
is treated by Sphinx as metadata of the extension [1].

With metadata "parallel_read_safe = True" a extension is marked as
save for "parallel reading of source". This is needed if you want
build in parallel with N processes. E.g.:

  make SPHINXOPTS=-j4 htmldocs

will no longer log warnings like:

  WARNING: the foobar extension does not declare if it is safe for
  parallel reading, assuming it isn't - please ask the extension author
  to check and make it explicit.

Add metadata to extensions:

* kernel-doc
* flat-table
* kernel-include

[1] http://www.sphinx-doc.org/en/stable/extdev/#extension-metadata

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Tested-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-09-01 08:19:02 -06:00
Markus Heiser
556aa6d5d9 doc-rst: moved *duplicate* warnings to nitpicky mode
Moved the *duplicate C object description* warnings for function
declarations in the nitpicky mode. In nitpick mode, you can suppress
those warnings (e.g. ioctl) with::

  nitpicky = True
  nitpick_ignore = [
      ("c:func", "ioctl"),
  ]

See Sphinx documentation for the config values for ``nitpick`` and
``nitpick_ignore`` [1].

With this change all the ".. cpp:function:: int ioctl(..)" descriptions
(found in the media book) can be migrated to ".. c:function:: int
ioctl(..)", without getting any warnings. E.g.::

  .. cpp:function:: int ioctl( int fd, int request, struct cec_event *argp )

  .. c:function:: int ioctl( int fd, int request, struct cec_event *argp )

The main effect, is that we get those *CPP-types* back into Sphinx's C-
namespace and we need no longer to distinguish between c/cpp references,
when we refer a function like the ioctl.

[1] http://www.sphinx-doc.org/en/stable/config.html?highlight=nitpick#confval-nitpicky

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-08-22 15:19:38 -06:00
Markus Heiser
2c645cd7c4 doc-rst:c-domain: ref-name of a function declaration
Add option 'name' to the "c:function:" directive.  With option 'name'
the ref-name of a function can be modified. E.g.::

    .. c:function:: int ioctl( int fd, int request )
       :name: VIDIOC_LOG_STATUS

The func-name (e.g. ioctl) remains in the output but the ref-name
changed from ``ioctl`` to ``VIDIOC_LOG_STATUS``. The index entry for
this function is also changed to ``VIDIOC_LOG_STATUS`` and the function
can now referenced by::

    :c:func:`VIDIOC_LOG_STATUS`

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-08-22 15:19:22 -06:00
Markus Heiser
e8f5c617f2 doc-rst: add boilerplate to customize c-domain
Add a sphinx-extension to customize the sphinx c-domain.  No functional
changes right yet, just the boilerplate code.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
[ jc: coding-style tweak ]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-08-22 15:19:15 -06:00
Mauro Carvalho Chehab
999d998eee docs-rst: parse-heraders.pl: escape LaTeX characters
Let's escape the LaTeX characters, to avoid troubles when
outputing them.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-08-18 16:42:07 -06:00
Markus Heiser
606b9ac81a doc-rst: generic way to build only sphinx sub-folders
Add a generic way to build only a reST sub-folder with or
without a individual *build-theme*.

* control *sub-folders* by environment SPHINXDIRS
* control *build-theme* by environment SPHINX_CONF

Folders with a conf.py file, matching $(srctree)/Documentation/*/conf.py
can be build and distributed *stand-alone*. E.g. to compile only the
html of 'media' and 'gpu' folder use::

  make SPHINXDIRS="media gpu" htmldocs

To use an additional sphinx-build configuration (*build-theme*) set the
name of the configuration file to SPHINX_CONF. E.g. to compile only the
html of 'media' with the *nit-picking* build use::

  make SPHINXDIRS=media SPHINX_CONF=conf_nitpick.py htmldocs

With this, the Documentation/conf.py is read first and updated with the
configuration values from the Documentation/media/conf_nitpick.py.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-08-14 11:51:51 -06:00
Mauro Carvalho Chehab
7e5b7d1b3a Merge branch 'docs-next' of git://git.lwn.net/linux into topic/docs-next
* 'docs-next' of git://git.lwn.net/linux:
  doc-rst: kernel-doc: fix handling of address_space tags
  Revert "doc/sphinx: Enable keep_warnings"
  doc-rst: kernel-doc directive, fix state machine reporter
  docs: deprecate kernel-doc-nano-HOWTO.txt
  doc/sphinx: Enable keep_warnings
  Documentation: add watermark_scale_factor to the list of vm systcl file
  kernel-doc: Fix up warning output
  docs: Get rid of some kernel-documentation warnings
2016-07-23 07:55:59 -03:00
Markus Heiser
f42ddca7be doc-rst: kernel-doc directive, fix state machine reporter
Add a reporter replacement that assigns the correct source name and line
number to a system message, as recorded in a ViewList.

[1] http://mid.gmane.org/CAKMK7uFMQ2wOp99t-8v06Om78mi9OvRZWuQsFJD55QA20BB3iw@mail.gmail.com

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Tested-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-07-20 16:51:12 -06:00
Mauro Carvalho Chehab
fb6fc6c9ac doc-rst: parse-headers: remove trailing spaces
The function that replace references add a "\ " at the end of
references, to avoid the ReST markup parser to not identify
them as references. That works fine except for the end of lines,
as a sequence of { '\', ' ', '\n' } characters makes Sphinx
to ignore the end of line. So, strip those escape/spaces at the
end of lines.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-09 09:42:31 -03:00
Markus Heiser
037763785a doc-rst: add kernel-include directive
The kernel-include directive is needed to include the auto generated rst
content from a build (pre-) process. E.g. the linux_tv Makefile
generates intermediate reST-files from header files. Since there is a O=
option:

  make O=dir [targets] Locate all output files in "dir"

We need to include intermediate reST files from arbitrary (O=/tmp/foo)
locations:

The 'kernel-include' reST-directive is a replacement for the 'include'
directive. The 'kernel-include' directive expand environment variables
in the path name and allows to include files from arbitrary locations.

.. hint::

  Including files from arbitrary locations (e.g. from '/etc') is a
  security risk for builders. This is why the 'include' directive from
  docutils *prohibit* pathnames pointing to locations *above* the
  filesystem tree where the reST document with the include directive is
  placed.

Substrings of the form $name or ${name} are replaced by the value of
environment variable name. Malformed variable names and references to
non-existing variables are left unchanged.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-08 09:31:04 -03:00
Mauro Carvalho Chehab
153234c47c doc-rst: parse-headers: don't do substituition references
Add one extra escape character to avoid those warnings:
	Documentation/linux_tv/videodev2.h.rst:6: WARNING: Inline substitution_reference start-string without end-string.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 15:13:17 -03:00
Mauro Carvalho Chehab
526b884831 doc-rst: parse-headers: add an option to ignore enum symbols
At videodev2.h, we have hundreds of symbols that don't
currently have a reference yet. Let's ignore for how, while
we don't improve those cross-refs.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 15:13:07 -03:00
Mauro Carvalho Chehab
034e6c8e72 doc-rst: parse-headers: better handle comments at the source code
We should not let comments to mangle with the symbols
parsing. Unfortunately, videodev2.h has lots of those
in the middle of enums and structs. So, we need to improve
our parser to discard them.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 15:12:48 -03:00
Mauro Carvalho Chehab
4ff916a0c9 doc-rst: parse-headers: fix multiline typedef handler
The typedef handler should do two things to be generic:
  1) parse typedef enums;
  2) accept both possible syntaxes:
	 typedef struct foo { .. } foo_t;
	 typedef struct { .. } foo_t;

Unfortunately, this is needed to parse some legacy DVB
files, like dvb/audio.h.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 08:12:57 -03:00
Mauro Carvalho Chehab
6c4c7dadb4 doc-rst: parse-headers: better handle typedefs
When typedef is used on its multiline format, we need to
also parse enum and struct in the same line.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 07:22:20 -03:00
Mauro Carvalho Chehab
9c80c74563 doc-rst: parse-headers: be more formal about the valid symbols
Be more formal about the valid symbols that are expected by
the parser, to match what c language expects.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 07:08:32 -03:00
Mauro Carvalho Chehab
9afe51129b doc-rst: fix parsing comments and '{' on a separate line
The dmx.h header has two things that causes the parser to
break while handling enums:
 per-header enums and the '{' starts on a new line

Both makes the parser to get lexical marks to be detected
as if they were symbols.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 06:54:00 -03:00
Mauro Carvalho Chehab
7d95fa8d66 doc-dst: parse-headers: highlight deprecated comments
When something is deprecated, highlight it, as we want it
to be clearer to the reader.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 06:36:46 -03:00
Mauro Carvalho Chehab
6fe79d1ede doc-rst: parse-headers: improve delimiters to detect symbols
As we had to escape the symbols for the ReST markup to not do
the wrong thing, the logic to discover start/end of strings
are not trivial. Improve the end delimiter detection, in order
to highlight more occurrences of the strings.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 06:36:46 -03:00
Mauro Carvalho Chehab
dabf8be33b doc-rst: add parse-headers.pl script
This script parses a header file and converts it into a
parsed-literal block, creating references for ioctls,
defines, typedefs, enums and structs.

It also allow an external file to modify the rules, in
order to fix the expressions.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 05:53:06 -03:00
Markus Heiser
0249a76448 doc-rst: flat-table directive - initial implementation
Implements the reST flat-table directive.

The ``flat-table`` is a double-stage list similar to the ``list-table`` with
some additional features:

* column-span: with the role ``cspan`` a cell can be extended through
  additional columns

* row-span: with the role ``rspan`` a cell can be extended through
  additional rows

* auto span rightmost cell of a table row over the missing cells on the right
  side of that table-row.  With Option ``:fill-cells:`` this behavior can
  changed from *auto span* to *auto fill*, which automaticly inserts (empty)

list tables

  The *list tables* formats are double stage lists. Compared to the
  ASCII-art they migth be less comfortable for readers of the
  text-files. Their advantage is, that they are easy to create/modify
  and that the diff of a modification is much more meaningfull, because
  it is limited to the modified content.

The initial implementation was taken from the sphkerneldoc project [1]

[1] https://github.com/return42/sphkerneldoc/commits/master/scripts/site-python/linuxdoc/rstFlatTable.py

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
[jc: fixed typos and misspellings in the docs]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-06-30 12:58:33 -06:00
Jani Nikula
03d35d9ec4 Documentation/sphinx: add support for specifying extra export files
Let the user specify file patterns where to look for the EXPORT_SYMBOLs
in addition to the file with kernel-doc comments. This is directly based
on the -export-file FILE option added to kernel-doc in "kernel-doc: add
support for specifying extra files for EXPORT_SYMBOLs", but we extend
that with globbing patterns in the Sphinx extension.

The file patterns are added as options to the :export: and :internal:
arguments of the kernel-doc directive. For example, to extract the
documentation of exported functions from include/net/mac80211.h:

.. kernel-doc:: include/net/mac80211.h
   :export: net/mac80211/*.c

Without the file pattern, no exported functions would be found, as the
EXPORT_SYMBOLs are placed in the various source files under
net/mac80211.

The matched files are also added as dependencies on the document in
Sphinx, as they may affect the output. This is one of the reasons to do
the globbing in the Sphinx extension instead of in scripts/kernel-doc.

The file pattern remains optional, and is not needed if the kernel-doc
comments and EXPORT_SYMBOLs are placed in the source file passed in as
the main argument to the kernel-doc directive. This is the most common
case across the kernel source tree.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-10 16:46:55 +03:00
Jani Nikula
057de5c4dd Documentation/sphinx: use a more sensible string split in kernel-doc extension
Using the default str.split doesn't return empty strings like the
current version does.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-10 11:29:21 +03:00
Jani Nikula
06173fe33a Documentation/sphinx: remove unnecessary temporary variable
Leftover cruft. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-10 11:29:20 +03:00
Daniel Vetter
d90368f2fa doc/sphinx: Track line-number of starting blocks
Design is pretty simple: kernel-doc inserts breadcrumbs with line
numbers, and sphinx picks them up. At first I went with a sphinx
comment, but inserting those at random places seriously upsets the
parser, and must be filtered. Hence why this version now uses "#define
LINEO " since one of these ever escape into output it's pretty clear
there is a bug.

It seems to work well, and at least the 2-3 errors where sphinx
complained about something that was not correct in kernel-doc text the
line numbers matched up perfectly.

v2: Instead of noodling around in the parser state machine, create
a ViewList and parse it ourselves. This seems to be the recommended
way, per Jani's suggestion.

v3:
- Split out ViewList pach. Splitting the kernel-doc changes from the
  sphinx ones isn't possible, since emitting the LINENO lines wreaks
  havoc with the rst formatting. We must filter them.

- Improve the regex per Jani's suggestions, and compile it just once
  for speed.

- Now that LINENO lines are eaten, also add them to function parameter
  descriptions. Much less content and offset than for in-line struct
  member descriptions, but still nice to know which exact continuation
  line upsets sphinx.

- Simplify/clarify the line +/-1 business a bit.

v4: Split out the scripts/kernel-doc changes and make line-numbers
opt-in, as suggested by Jani.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-04 11:35:59 +03:00
Daniel Vetter
16e161c8c6 doc/sphinx: Stop touching state_machine internals
Instead of just forcefully inserting our kernel-doc input and letting
the state machine stumble over it the recommended way is to create
ViewList, parse that and then return the list of parsed nodes.

Suggested by Jani.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-03 11:31:37 +03:00
Daniel Vetter
9cc1a44c03 doc/sphinx: Pass right filename as source
With this error output becomes almost readable. The line numbers are
still totally bonghits, but that's a lot harder to pull out of
kerneldoc. We'd essentially have to insert some special markers in the
kernel-doc output, split the output along these markers and then
insert each block separately using

     state_machine.insert_input(block, source, first_line)

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-01 12:28:31 +03:00
Jani Nikula
ba35018593 Documentation/sphinx: fix kernel-doc extension on python3
Reconcile differences between python2 and python3 on dealing with
stdout, stderr from Popen. This fixes "name 'unicode' is not defined"
errors on python3. We'll need to try to keep the extension working on
both python-sphinx and python3-sphinx so we don't need two copies.

Reported-and-tested-by: Marius Vlad <marius.c.vlad@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-01 11:06:58 +03:00
Jani Nikula
30ca7aaf27 Documentation/sphinx: nicer referencing of struct in docbook->rst conversion
Add "struct" in the label of the reference.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:54 +03:00
Jani Nikula
2e83ecb834 sphinx: update docbook->rst conversion script match C domain spec
Function references should include the parens (), struct references
should not include "struct".

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:53 +03:00
Jonathan Corbet
89a66d7610 sphinx: cheesy script to convert .tmpl files
This script uses pandoc to convert existing DocBook template files to RST
templates.  A couple of sed scripts are need to massage things both before
and after the conversion, but the result is then usable with no hand
editing.

[Jani: Change usage to tmplcvt <in> <out>. Fix escaping for docproc
directives. Add support the new kernel-doc extension.]

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:53 +03:00
Jani Nikula
c56de1db54 Documentation/sphinx: add Sphinx kernel-doc directive extension
Add an extension to handle kernel-doc directives, to call kernel-doc
according to the arguments and parameters given to the reStructuredText
directive.

The syntax for the kernel-doc directive is:

.. kernel-doc:: FILENAME
   :export:
   :internal:
   :functions: FUNCTION [FUNCTION ...]
   :doc: SECTION TITLE

Of the directive options export, internal, functions, and doc, currently
only one option may be given at a time.

The FILENAME is relative from the kernel source tree root.

The extension notifies Sphinx about the document dependency on FILENAME,
causing the document to be rebuilt when the file has been changed.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:52 +03:00