Skip to content

Commit

Permalink
doc: auto toctree for examples and microservices (#130)
Browse files Browse the repository at this point in the history
Instead of manually maintaining the headings and toctree for examples
and microservices, build the toctree using a script during doc build
time that scans for directories in GenAIExamples and GenAIComps/comps.

This way, if new examples or comps show up (or are renamed) they're
automatically added to these index documents for github.io
publishing.

Signed-off-by: David B. Kinder <[email protected]>
  • Loading branch information
dbkinder authored Sep 14, 2024
1 parent 86d44ef commit d97aab5
Show file tree
Hide file tree
Showing 4 changed files with 78 additions and 184 deletions.
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ content:
# $(Q)rsync $(RSYNC_OPTS) ../docs/* $(SOURCEDIR)
$(Q)find $(SOURCEDIR) -type f -empty -name "README.md" -delete
$(Q)scripts/fix-github-md-refs.sh $(SOURCEDIR)
$(Q)scripts/maketoc.sh $(SOURCEDIR)


html: content
Expand Down
186 changes: 3 additions & 183 deletions examples/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,6 @@ We're building this documentation from content in the
/GenAIExamples/README
/GenAIExamples/*

----

**Example Applications Table of Contents**

.. rst-class:: rst-columns
Expand All @@ -39,185 +37,7 @@ We're building this documentation from content in the

----

AgentQnA Application
********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/AgentQnA/*
/GenAIExamples/AgentQnA/**/*

AudioQnA Application
********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/AudioQnA/*
/GenAIExamples/AudioQnA/**/*

ChatQnA Application
*******************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/ChatQnA/*
/GenAIExamples/ChatQnA/**/*

Code Generation Application
***************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/CodeGen/*
/GenAIExamples/CodeGen/**/*

Code Translation Application
****************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/CodeTrans/*
/GenAIExamples/CodeTrans/**/*

Document Index Retriever Application
************************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/DocIndexRetriever/*
/GenAIExamples/DocIndexRetriever/**/*


Document Summarization Application
**********************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/DocSum/*
/GenAIExamples/DocSum/**/*

FaqGen Application
*********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/FaqGen/*
/GenAIExamples/FaqGen/**/*

Instruction Tuning Application
******************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/InstructionTuning/*
/GenAIExamples/InstructionTuning/**/*


ProductivitySuite Application
*****************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/ProductivitySuite/*
/GenAIExamples/ProductivitySuite/**/*

Rerank Model Fine Tuning
************************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/RerankFinetuning/*
/GenAIExamples/RerankFinetuning/**/*


SearchQnA Application
*********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/SearchQnA/*
/GenAIExamples/SearchQnA/**/*

Translation Application
***********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/Translation/*
/GenAIExamples/Translation/**/*

VideoQnA Application
***********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:

/GenAIExamples/VideoQnA/*
/GenAIExamples/VideoQnA/**/*

VisualQnA Application
*********************

.. rst-class:: rst-columns

.. toctree::
:maxdepth: 1
:glob:
.. comment This include file is generated in the Makefile during doc build
time from all the directories found in the GenAIExamples top level directory
/GenAIExamples/VisualQnA/*
/GenAIExamples/VisualQnA/**/*
.. include:: examples.txt
15 changes: 14 additions & 1 deletion microservices/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,18 @@ We're building this microservices documentation from content in the

/GenAIComps/README
/GenAIComps/*
/GenAIComps/**/*

**Microservices Table of Contents**

.. rst-class:: rst-columns

.. contents::
:local:
:depth: 1

----

.. comment This include file is generated in the Makefile during doc build
time from all the directories found in the GenAIComps/comps directory
.. include:: microservices.txt
60 changes: 60 additions & 0 deletions scripts/maketoc.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/bin/bash
# Copyright (C) 2024 Intel Corporation.
# SPDX-License-Identifier: Apache-2.0

# For all directories, create a H2 title with the directory name and a
# .toctree directive with the repo name and
# those directory names, something like this:
#
# AudioQnA Application
# ********************
#
# .. toctree::
# :maxdepth: 1
# :glob:
#
# /GenAIExamples/AudioQnA/*
# /GenAIExamples/AudioQnA/**/*
#
# ls -d1 */ returns something like this:
#
# AgentQnA/
# AudioQnA/
# ChatQnA/
# CodeGen/
# CodeTrans/
# DocIndexRetriever/
#
#
# Create a title based on the directory name and print it.
# Use a gsub to turn every character in the title into an * for the
# heading underline and print it.

cd ../GenAIExamples

ls -d1 */ | \
awk \
-v repo="GenAIExamples" \
-e '{dirname=substr($0,1,length($0)-1); title=dirname " Application"; \
print title;gsub(/./,"*",title); print title; \
print "\n.. rst-class:: rst-columns\n\n.. toctree::\n :maxdepth: 1\n :glob:\n\n /" \
repo "/" dirname "/*\n /" \
repo "/" dirname "/**/*\n";}' > ../docs/_build/rst/examples/examples.txt

#
# The components directory names in GenAIComps/comps are all lowercase, so uppercase
# just the first character for the title.
#

cd ../GenAIComps/comps

ls -d1 [a-zA-Z]*/ | \
awk \
-v repo="GenAIComps" \
-e '{dirname=substr($0,1,length($0)-1); title=toupper(substr(dirname,1,1)) substr(dirname,2) " Microservice"; \
print title;gsub(/./,"*",title); print title; \
print "\n.. rst-class:: rst-columns\n\n.. toctree::\n :maxdepth: 1\n :glob:\n\n /" \
repo "/comps/" dirname "/*\n /" \
repo "/comps/" dirname "/**/*\n";}' > ../../docs/_build/rst/microservices/microservices.txt


0 comments on commit d97aab5

Please sign in to comment.