[gtk-doc] design: more planning updates for no-docbook future

[Stefan Sauer <stefkost src gnome org>]

commit 31d80407ecc4a3e74f19a091252519f0c5d708b2
Author: Stefan Sauer <ensonic users sf net>
Date:   Wed Nov 8 20:00:00 2017 +0100

    design: more planning updates for no-docbook future

 TODO               |   17 +++++++----------
 doc/design-2.x.txt |   38 +++++++++++++++++++++++---------------
 2 files changed, 30 insertions(+), 25 deletions(-)
---
diff --git a/TODO b/TODO
index 9049f6b..d881955 100644
--- a/TODO
+++ b/TODO
@@ -67,11 +67,6 @@ https://bugzilla.gnome.org/show_bug.cgi?id=646094
 * expand urls (needds more work, see gtkdoc-mkdb : ExpandAbbreviations)
 
 
-= Testing =
-cd test/gobject
-diff -u --exclude="Makefile*" docs docs-tmpl | diffstat
-
-
 = Running =
 gtk-doc is using a makefile with several targets to get from sources to docs. It
 uses makefile variables for configuration.
@@ -80,15 +75,15 @@ in the right order (ev. by importing them as modules). This could handle a few
 things nicer that the makefiles don't do well. This would also make it easy to
 run it manually or integrate into other build systems.
 
+
 = Intermediate files =
-Can we change the intermediate files into a perl-module? So that all we need to
-do is to import it. Ideally we have just one outfile for each tool:
-gtkdoc-scan     -> scan.pm (decl.txt, decl-list.txt, types.txt, sections.txt)
-gtkdoc-scangobj -> scanobj.pm (interfacs, prerequisites, ...)
+Can we change the intermediate files into e.g. json or yaml so that we can just
+load it from python?
 
 Can we drop the decl-list.txt file (or rename to make sure this is what a
 generated section.txt would look like)
 
+
 = Issues =
 * gtkdoc-fixxref makefile targets use $HTML_DIR
   * HTML_DIR: The directory where gtk-doc generated documentation is installed
@@ -101,7 +96,7 @@ generated section.txt would look like)
 * gtkdoc does not know about module versions
   * this is causing troubles when multiple library versions are installed
     (gtk+-{2,3}, gstreamer-{0.8,0.10,1.0}
-  * right now gtk-doc is xreffing against any <module>/index.sgml found
+  * right now gtk-doc is xreffing against any <module>/*.devhelp2 found
   * just using the one with a higher number won't work
 
 
@@ -143,6 +138,7 @@ libs can benefit from the extensions too.
   * we might also need to reflow some things, as gtk-doc.xsl also runs the devhelp/devhelp2 generation
     - but then the urls in the devhelp file, refer to the chunked html anyway
 
+
 = Warnings =
 Bugzilla has some requests for extra warnings. We should support a common
 commandline option(s) in all tools to enable/disable the warnings. The makefiles
@@ -169,6 +165,7 @@ more warnings:
     from the symbol.
   - possible xrefs (e.g. adding a # or () would make it a successful xref)
 
+
 = GIR =
 == scanning ==
 * ideas
diff --git a/doc/design-2.x.txt b/doc/design-2.x.txt
index 6eb5fb8..c4fd7c2 100644
--- a/doc/design-2.x.txt
+++ b/doc/design-2.x.txt
@@ -78,29 +78,37 @@ If the ID contain a ':' xml processors believe it is using a namespace.
 
 === docbook -> markdown ===
 The main performance culprit is the use of docbook (tools). Also writing docbook
-in source comments is cumbersome. We already support lots of markdown, so we'd
+in source comments is cumbersome. Finally the docbook toolchain is a heavy
+dependency and harms portability. We already support lots of markdown, so we'd
 like to have a pure markdown based workflow.
 
 For that we'd like to provide a migration path:
-- convert existing docbook files to markdown (e.g. pandoc)
+- convert existing docbook files to markdown (e.g. using pandoc)
 - convert/identify comments using docbook to markdown
 
-Next we need a replacement for gtkdoc-mkdb: gtkdoc-mkmd to create extract
-markdown from  sources and generate support files (index files). We also need to
-patch gtkdoc-{mkdb/pdf} to run respective tools (e.g. pandoc).
+Next we would replace gtkdoc-mkdb+gtkdoc-fixxref with gtkdoc-mkhtml2. This would
+output html directly.
+
+We could change gtkdoc-mkpdf to use https://wkhtmltopdf.org/. For man-pages we
+can use https://rtomayko.github.io/ronn/ronn.1.html.
+The devhelp2 files would be output directly from gtkdoc-mkhtml2.
 
 We can enable such a toolchain via the configure flavors option.
 
-TODO:
-- markdown does not define a way to aggregate fragments into a large book
-  - we can only concat all the files in a stable order
+These would be the steps to do this:
+1.) [unassigned] write the docbook comment migation tool:
+This would convert the comments in the given source file. One would make a copy
+of the docs/xml dir, run the migration tool for some sources, rebuild the docs
+and compare the new xml. If all good, submit the changes.
 
-As a first step we need to make sure we only render the html files with docbook.
-Right now we also produce the devhelp files using xslt (see devhelp2.xsl). We
-should be able to produce the devhelp files from gtkdoc-mkdb directly. We can
-generate this file outside of the html dir (and adjust the make install target).
-The tricky part is that we might need to scan the static content files to be
-able to generate the same toc.
-A nice extra benefit would be that this should already make things faster.
+2.) [unassigned] create the plumbing for the new tool chain
+Add the new makefile flavour to run gtkdoc-mkhtml2 instead. Add --flavour
+options for gtkdocize. Create a stub  gtkdoc-mkhtml2 tool.
 
+3.) [in progress] refactor gtkdoc/mkdb.py to exctract reusable code
+- gtkdoc/md_to_db.py
+  - only have the parse there
 
+4.) [unassigned] write gtkdoc/mkhtml2.py
+- create templates from the current html for the various page types (refentry,
+  index, ...).
\ No newline at end of file