Commit Graph

124 Commits

Author SHA1 Message Date
Mauro Carvalho Chehab
fb6fc6c9ac doc-rst: parse-headers: remove trailing spaces
The function that replace references add a "\ " at the end of
references, to avoid the ReST markup parser to not identify
them as references. That works fine except for the end of lines,
as a sequence of { '\', ' ', '\n' } characters makes Sphinx
to ignore the end of line. So, strip those escape/spaces at the
end of lines.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-09 09:42:31 -03:00
Markus Heiser
037763785a doc-rst: add kernel-include directive
The kernel-include directive is needed to include the auto generated rst
content from a build (pre-) process. E.g. the linux_tv Makefile
generates intermediate reST-files from header files. Since there is a O=
option:

  make O=dir [targets] Locate all output files in "dir"

We need to include intermediate reST files from arbitrary (O=/tmp/foo)
locations:

The 'kernel-include' reST-directive is a replacement for the 'include'
directive. The 'kernel-include' directive expand environment variables
in the path name and allows to include files from arbitrary locations.

.. hint::

  Including files from arbitrary locations (e.g. from '/etc') is a
  security risk for builders. This is why the 'include' directive from
  docutils *prohibit* pathnames pointing to locations *above* the
  filesystem tree where the reST document with the include directive is
  placed.

Substrings of the form $name or ${name} are replaced by the value of
environment variable name. Malformed variable names and references to
non-existing variables are left unchanged.

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-08 09:31:04 -03:00
Mauro Carvalho Chehab
153234c47c doc-rst: parse-headers: don't do substituition references
Add one extra escape character to avoid those warnings:
	Documentation/linux_tv/videodev2.h.rst:6: WARNING: Inline substitution_reference start-string without end-string.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 15:13:17 -03:00
Mauro Carvalho Chehab
526b884831 doc-rst: parse-headers: add an option to ignore enum symbols
At videodev2.h, we have hundreds of symbols that don't
currently have a reference yet. Let's ignore for how, while
we don't improve those cross-refs.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 15:13:07 -03:00
Mauro Carvalho Chehab
034e6c8e72 doc-rst: parse-headers: better handle comments at the source code
We should not let comments to mangle with the symbols
parsing. Unfortunately, videodev2.h has lots of those
in the middle of enums and structs. So, we need to improve
our parser to discard them.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 15:12:48 -03:00
Mauro Carvalho Chehab
4ff916a0c9 doc-rst: parse-headers: fix multiline typedef handler
The typedef handler should do two things to be generic:
  1) parse typedef enums;
  2) accept both possible syntaxes:
	 typedef struct foo { .. } foo_t;
	 typedef struct { .. } foo_t;

Unfortunately, this is needed to parse some legacy DVB
files, like dvb/audio.h.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 08:12:57 -03:00
Mauro Carvalho Chehab
6c4c7dadb4 doc-rst: parse-headers: better handle typedefs
When typedef is used on its multiline format, we need to
also parse enum and struct in the same line.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 07:22:20 -03:00
Mauro Carvalho Chehab
9c80c74563 doc-rst: parse-headers: be more formal about the valid symbols
Be more formal about the valid symbols that are expected by
the parser, to match what c language expects.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 07:08:32 -03:00
Mauro Carvalho Chehab
9afe51129b doc-rst: fix parsing comments and '{' on a separate line
The dmx.h header has two things that causes the parser to
break while handling enums:
 per-header enums and the '{' starts on a new line

Both makes the parser to get lexical marks to be detected
as if they were symbols.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 06:54:00 -03:00
Mauro Carvalho Chehab
7d95fa8d66 doc-dst: parse-headers: highlight deprecated comments
When something is deprecated, highlight it, as we want it
to be clearer to the reader.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 06:36:46 -03:00
Mauro Carvalho Chehab
6fe79d1ede doc-rst: parse-headers: improve delimiters to detect symbols
As we had to escape the symbols for the ReST markup to not do
the wrong thing, the logic to discover start/end of strings
are not trivial. Improve the end delimiter detection, in order
to highlight more occurrences of the strings.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 06:36:46 -03:00
Mauro Carvalho Chehab
dabf8be33b doc-rst: add parse-headers.pl script
This script parses a header file and converts it into a
parsed-literal block, creating references for ioctls,
defines, typedefs, enums and structs.

It also allow an external file to modify the rules, in
order to fix the expressions.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
2016-07-07 05:53:06 -03:00
Markus Heiser
0249a76448 doc-rst: flat-table directive - initial implementation
Implements the reST flat-table directive.

The ``flat-table`` is a double-stage list similar to the ``list-table`` with
some additional features:

* column-span: with the role ``cspan`` a cell can be extended through
  additional columns

* row-span: with the role ``rspan`` a cell can be extended through
  additional rows

* auto span rightmost cell of a table row over the missing cells on the right
  side of that table-row.  With Option ``:fill-cells:`` this behavior can
  changed from *auto span* to *auto fill*, which automaticly inserts (empty)

list tables

  The *list tables* formats are double stage lists. Compared to the
  ASCII-art they migth be less comfortable for readers of the
  text-files. Their advantage is, that they are easy to create/modify
  and that the diff of a modification is much more meaningfull, because
  it is limited to the modified content.

The initial implementation was taken from the sphkerneldoc project [1]

[1] https://github.com/return42/sphkerneldoc/commits/master/scripts/site-python/linuxdoc/rstFlatTable.py

Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
[jc: fixed typos and misspellings in the docs]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2016-06-30 12:58:33 -06:00
Jani Nikula
03d35d9ec4 Documentation/sphinx: add support for specifying extra export files
Let the user specify file patterns where to look for the EXPORT_SYMBOLs
in addition to the file with kernel-doc comments. This is directly based
on the -export-file FILE option added to kernel-doc in "kernel-doc: add
support for specifying extra files for EXPORT_SYMBOLs", but we extend
that with globbing patterns in the Sphinx extension.

The file patterns are added as options to the :export: and :internal:
arguments of the kernel-doc directive. For example, to extract the
documentation of exported functions from include/net/mac80211.h:

.. kernel-doc:: include/net/mac80211.h
   :export: net/mac80211/*.c

Without the file pattern, no exported functions would be found, as the
EXPORT_SYMBOLs are placed in the various source files under
net/mac80211.

The matched files are also added as dependencies on the document in
Sphinx, as they may affect the output. This is one of the reasons to do
the globbing in the Sphinx extension instead of in scripts/kernel-doc.

The file pattern remains optional, and is not needed if the kernel-doc
comments and EXPORT_SYMBOLs are placed in the source file passed in as
the main argument to the kernel-doc directive. This is the most common
case across the kernel source tree.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-10 16:46:55 +03:00
Jani Nikula
057de5c4dd Documentation/sphinx: use a more sensible string split in kernel-doc extension
Using the default str.split doesn't return empty strings like the
current version does.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-10 11:29:21 +03:00
Jani Nikula
06173fe33a Documentation/sphinx: remove unnecessary temporary variable
Leftover cruft. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-10 11:29:20 +03:00
Daniel Vetter
d90368f2fa doc/sphinx: Track line-number of starting blocks
Design is pretty simple: kernel-doc inserts breadcrumbs with line
numbers, and sphinx picks them up. At first I went with a sphinx
comment, but inserting those at random places seriously upsets the
parser, and must be filtered. Hence why this version now uses "#define
LINEO " since one of these ever escape into output it's pretty clear
there is a bug.

It seems to work well, and at least the 2-3 errors where sphinx
complained about something that was not correct in kernel-doc text the
line numbers matched up perfectly.

v2: Instead of noodling around in the parser state machine, create
a ViewList and parse it ourselves. This seems to be the recommended
way, per Jani's suggestion.

v3:
- Split out ViewList pach. Splitting the kernel-doc changes from the
  sphinx ones isn't possible, since emitting the LINENO lines wreaks
  havoc with the rst formatting. We must filter them.

- Improve the regex per Jani's suggestions, and compile it just once
  for speed.

- Now that LINENO lines are eaten, also add them to function parameter
  descriptions. Much less content and offset than for in-line struct
  member descriptions, but still nice to know which exact continuation
  line upsets sphinx.

- Simplify/clarify the line +/-1 business a bit.

v4: Split out the scripts/kernel-doc changes and make line-numbers
opt-in, as suggested by Jani.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-04 11:35:59 +03:00
Daniel Vetter
16e161c8c6 doc/sphinx: Stop touching state_machine internals
Instead of just forcefully inserting our kernel-doc input and letting
the state machine stumble over it the recommended way is to create
ViewList, parse that and then return the list of parsed nodes.

Suggested by Jani.

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-03 11:31:37 +03:00
Daniel Vetter
9cc1a44c03 doc/sphinx: Pass right filename as source
With this error output becomes almost readable. The line numbers are
still totally bonghits, but that's a lot harder to pull out of
kerneldoc. We'd essentially have to insert some special markers in the
kernel-doc output, split the output along these markers and then
insert each block separately using

     state_machine.insert_input(block, source, first_line)

Cc: Jani Nikula <jani.nikula@intel.com>
Cc: linux-doc@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-01 12:28:31 +03:00
Jani Nikula
ba35018593 Documentation/sphinx: fix kernel-doc extension on python3
Reconcile differences between python2 and python3 on dealing with
stdout, stderr from Popen. This fixes "name 'unicode' is not defined"
errors on python3. We'll need to try to keep the extension working on
both python-sphinx and python3-sphinx so we don't need two copies.

Reported-and-tested-by: Marius Vlad <marius.c.vlad@intel.com>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-06-01 11:06:58 +03:00
Jani Nikula
30ca7aaf27 Documentation/sphinx: nicer referencing of struct in docbook->rst conversion
Add "struct" in the label of the reference.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:54 +03:00
Jani Nikula
2e83ecb834 sphinx: update docbook->rst conversion script match C domain spec
Function references should include the parens (), struct references
should not include "struct".

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:53 +03:00
Jonathan Corbet
89a66d7610 sphinx: cheesy script to convert .tmpl files
This script uses pandoc to convert existing DocBook template files to RST
templates.  A couple of sed scripts are need to massage things both before
and after the conversion, but the result is then usable with no hand
editing.

[Jani: Change usage to tmplcvt <in> <out>. Fix escaping for docproc
directives. Add support the new kernel-doc extension.]

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:53 +03:00
Jani Nikula
c56de1db54 Documentation/sphinx: add Sphinx kernel-doc directive extension
Add an extension to handle kernel-doc directives, to call kernel-doc
according to the arguments and parameters given to the reStructuredText
directive.

The syntax for the kernel-doc directive is:

.. kernel-doc:: FILENAME
   :export:
   :internal:
   :functions: FUNCTION [FUNCTION ...]
   :doc: SECTION TITLE

Of the directive options export, internal, functions, and doc, currently
only one option may be given at a time.

The FILENAME is relative from the kernel source tree root.

The extension notifies Sphinx about the document dependency on FILENAME,
causing the document to be rebuilt when the file has been changed.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
2016-05-30 13:38:52 +03:00