Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Namnn/need per id json #960

Merged
merged 91 commits into from
Sep 6, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
eb546d4
gen needs.json per needs.id and command some service git to bypass test
nhatnamnguyengtvthcm Jul 7, 2023
600ad3f
update review 11/07
nhatnamnguyengtvthcm Jul 12, 2023
0228b24
update review 11/07
nhatnamnguyengtvthcm Jul 12, 2023
9a698b0
update conf
nhatnamnguyengtvthcm Jul 14, 2023
3de0647
config variable for needs_id forder path
nhatnamnguyengtvthcm Jul 19, 2023
dbb52c0
update env config for needs per id
nhatnamnguyengtvthcm Jul 21, 2023
01320d4
add configuration for needs per id~
nhatnamnguyengtvthcm Jul 21, 2023
177d3e3
add configuration for needs per id~
nhatnamnguyengtvthcm Jul 21, 2023
efbe1fa
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
04f5019
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
be61481
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
103cc2f
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
274842f
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
1c2af5b
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
9136af8
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
26a18b8
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
ac7c5fe
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
618e1b0
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
8802a04
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
55e0cd3
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
6462295
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
dbb9f3f
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
13c1ca7
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
86bcfef
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
f42e9d7
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
f0ec457
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 10, 2023
724a367
change version build
nhatnamnguyengtvthcm Aug 21, 2023
7e5b7e4
change changelog file
nhatnamnguyengtvthcm Aug 21, 2023
86e343a
Removed esbonio for IDE support
haiyangToAI Feb 17, 2023
30cae4d
enable docker arm builds
cpolzer May 31, 2023
41c0368
🔧 Add .nox to gitignore
chrisjsewell Aug 2, 2023
bb83ac1
Update pyproject.toml
chrisjsewell Aug 3, 2023
c3ff8b6
Update pyproject.toml
chrisjsewell Aug 3, 2023
7ca8d6e
Update requirements.txt
chrisjsewell Aug 3, 2023
7680146
Update api.rst
chrisjsewell Aug 3, 2023
e969347
Added config allow unsafe filter for filter_func (#949)
haiyangToAI Aug 3, 2023
3935da6
🔧 Update pre-commit hooks (#956)
chrisjsewell Aug 4, 2023
e518929
Bump actions/checkout from 3.3.0 to 3.5.3
dependabot[bot] Jun 12, 2023
8a500e6
needs: Remove some of the extra IDs in the output
tim-nordell-nimbelink Nov 21, 2022
786db24
Added config option needs_report_dead_links (#937)
haiyangToAI Aug 4, 2023
624fd84
Raises version to 1.3.0
danwos Aug 16, 2023
ee77802
👌 Performance: Memoize Inline Parser
chrisjsewell Aug 11, 2023
cbc7bc0
fix linting
chrisjsewell Aug 16, 2023
ceae80e
Add to changelog
chrisjsewell Aug 16, 2023
4ee996e
Easier Sphinx-Needs docs builds
danwos Aug 18, 2023
b8fe439
Docker fix
danwos Aug 18, 2023
1b79d17
Removing docker context: ci
danwos Aug 18, 2023
ab99745
gen needs.json per needs.id and command some service git to bypass test
nhatnamnguyengtvthcm Jul 7, 2023
b9a0f99
update conf
nhatnamnguyengtvthcm Jul 14, 2023
f868ffe
update env config for needs per id
nhatnamnguyengtvthcm Jul 21, 2023
4985e9f
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
0c8e80f
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
18cbd4e
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
305ad77
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 10, 2023
d9a967d
change version build
nhatnamnguyengtvthcm Aug 21, 2023
a9edc4e
update change log
nhatnamnguyengtvthcm Aug 21, 2023
bd18a2b
change release version
nhatnamnguyengtvthcm Aug 21, 2023
6d50c27
♻️ Change `NeedsBuilder` format to `needs`
chrisjsewell Aug 21, 2023
b6819f4
Update changelog.rst
chrisjsewell Aug 21, 2023
588e4cf
gen needs.json per needs.id and command some service git to bypass test
nhatnamnguyengtvthcm Jul 7, 2023
5c30c12
update review 11/07
nhatnamnguyengtvthcm Jul 12, 2023
6fd7eba
fix-ci-lint-err
nhatnamnguyengtvthcm Aug 4, 2023
f85bb09
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 7, 2023
d474b40
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
aaf3869
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 9, 2023
1a288e0
add test-case and fix some setting for need per id builder
nhatnamnguyengtvthcm Aug 10, 2023
d8013c9
change version build
nhatnamnguyengtvthcm Aug 21, 2023
819e716
change changelog file
nhatnamnguyengtvthcm Aug 21, 2023
7aac5d3
Raises version to 1.3.0
danwos Aug 16, 2023
41609ff
Add to changelog
chrisjsewell Aug 16, 2023
4d88520
Easier Sphinx-Needs docs builds
danwos Aug 18, 2023
6f5e62e
update conf
nhatnamnguyengtvthcm Jul 14, 2023
6ff9b71
change version build
nhatnamnguyengtvthcm Aug 21, 2023
d1c46ac
rebase and merge version 1.3.0
nhatnamnguyengtvthcm Aug 21, 2023
c2ef673
merge version 1.3.0
nhatnamnguyengtvthcm Aug 21, 2023
5e8fc85
fix merge version 1.3.0
nhatnamnguyengtvthcm Aug 21, 2023
71988c4
fix merge version 1.3.0
nhatnamnguyengtvthcm Aug 21, 2023
a0c771b
move kogic in needs id builder into needsfile
nhatnamnguyengtvthcm Aug 25, 2023
4d06008
move kogic in needs id builder into needsfile
nhatnamnguyengtvthcm Aug 25, 2023
a5af64f
move kogic in needs id builder into needsfile
nhatnamnguyengtvthcm Aug 25, 2023
9077dca
using doc_needs_builder for test needs_id builder
nhatnamnguyengtvthcm Aug 28, 2023
025cfc5
mofify configuration.rst
nhatnamnguyengtvthcm Aug 28, 2023
dafb701
merge version 1.3.0
nhatnamnguyengtvthcm Aug 28, 2023
cf47fd4
change version needs_id builder
nhatnamnguyengtvthcm Aug 28, 2023
b84e4cc
fix test case for needs_id_builder
nhatnamnguyengtvthcm Aug 28, 2023
ef926b1
update version 1.3.0
nhatnamnguyengtvthcm Aug 30, 2023
0ac8a56
update version 1.3.0
nhatnamnguyengtvthcm Aug 30, 2023
3a10072
update version 1.3.0
nhatnamnguyengtvthcm Aug 30, 2023
49e91cb
update version 1.3.0
nhatnamnguyengtvthcm Aug 30, 2023
43689e5
refactor confiuration file and builder
nhatnamnguyengtvthcm Aug 31, 2023
ba544e7
add version 1.4.0 in needs_build_json_per_id_path configurations.rst
nhatnamnguyengtvthcm Sep 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,5 @@ Jörg Kreuzberger <[email protected]>
Duodu Randy <[email protected]>

Christian Wappler <[email protected]>

Nhat Nam Nguen <[email protected]>
nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved
18 changes: 18 additions & 0 deletions docs/builders.rst
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,21 @@ or
.. hint::

As an alternative, you can set the config option :ref:`needs_build_needumls` to export the needumls files during each build.


.. _needs_id_builder:

needs_id
--------
.. versionadded:: 0.1.30
nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved

The **needs_id** builder exports all found needs and selected filter results to a set json files of each need with the name is ``id`` of need.
danwos marked this conversation as resolved.
Show resolved Hide resolved

danwos marked this conversation as resolved.
Show resolved Hide resolved
The build creates a folder called :ref:``needs_build_json_per_id_path`` and all file json of each need inside the given build-folder.
danwos marked this conversation as resolved.
Show resolved Hide resolved

Usage
+++++

.. code-block:: bash

sphinx-build -b needs_id source_dir build_dir
4 changes: 3 additions & 1 deletion docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Released: under development
* Improvement: Configuration option :ref:`needs_debug_measurement` added, which creates a runtime report
for debugging purposes.
(`#917 <https://github.com/useblocks/sphinx-needs/pull/917>`_)

* Improvement: Builder :ref:`NeedsIdBuilder` added, which exports all needs to list json file for each need.
* Bugfix: Replace hardcoded `index` with config value `root_doc`.
(`#877 <https://github.com/useblocks/sphinx-needs/pull/877>`_)
* Bugfix: Fix unbounded memory usage in pickle environment.
Expand Down Expand Up @@ -889,3 +889,5 @@ custom css definitions you need to update them.
* Integrated interaction with the activated plantuml sphinx extension

* Added role **need** to create a reference to a need by giving the id


4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -379,8 +379,8 @@ def custom_defined_func():

# build needs.json to make permalinks work
needs_build_json = True
# build needs per id
needs_per_id = True
# build needs_json for every needs-id to make detail panel
needs_build_json_per_id = True
nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved
nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved
# Get and maybe set GitHub credentials for services.
# This is needed as the rate limit for not authenticated users is too low for the amount of requests we
# need to perform for this documentation
Expand Down
33 changes: 25 additions & 8 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2339,23 +2339,40 @@ Default value: True

Can be overwritten for each single need by setting :ref:`need_collapse`.

.. _needs_per_id_build_path:
.. _needs_build_json_per_id_path:

needs_per_id_build_path
~~~~~~~~~~~~~~~~
needs_build_json_per_id_path
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This option sets the location of the set of ``needs.json`` for every needs-id.

Default value: needs_id

Example:

.. _needs_per_id:
.. code-block:: python

needs_per_id
~~~~~~~~~~~~~~~~
needs_build_json_per_id_path = "needs_id"

Exports forder ``needs_per_id_build_path`` include all file ``needs.json`` for every needs-id.
.. hint::

The created ``needs_id`` folder gets stored in the ``outdir`` of the current builder. The final location is e.g. ``_build/needs_id``

This option works like needs_build_json.
.. _needs_build_json_per_id:

nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved
needs_build_json_per_id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document first needs_build_json_per_id then needs_build_json_per_id_path, as needs_build_json_per_id_path is based on needs_build_json_per_id

~~~~~~~~~~~~~~~~~~~~~~~
Builds list json file for each need, which have name is ``ID`` of need.
This option works like :ref:`needs_build_json`.

Default: False

Example:

.. code-block:: python

needs_build_json_per_id = False

.. hint::
The created single json file per need, located in :ref:`needs_build_json_per_id_path` folder. This is e.g ``_build/needs_id/abc_432.json``

12 changes: 7 additions & 5 deletions sphinx_needs/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@ def get_target_uri(self, _docname: str, _typ: Optional[str] = None) -> str:


def build_needs_json(app: Sphinx, _exception: Exception) -> None:

env = unwrap(app.env)

if not env.config.needs_build_json:
Expand Down Expand Up @@ -160,6 +159,9 @@ def build_needumls_pumls(app: Sphinx, _exception: Exception) -> None:


class NeedsIdBuilder(Builder):
danwos marked this conversation as resolved.
Show resolved Hide resolved

nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved
"""Json builder for needs, which creates one single json-file per need"""
nhatnamnguyengtvthcm marked this conversation as resolved.
Show resolved Hide resolved

name = "needs_id"
danwos marked this conversation as resolved.
Show resolved Hide resolved
format = "json"
Copy link
Member

@chrisjsewell chrisjsewell Aug 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
format = "json"
format = NeedsBuilder.format

The NeedsBuilder.format was changed in #978, I would suggest this would be the simplest way to keep them in-sync (and obviously the same for #961 and #962 cheers

file_suffix = ".txt"
Expand All @@ -172,10 +174,10 @@ def write_doc(self, docname: str, doctree: nodes.document) -> None:
def finish(self) -> None:
env = unwrap(self.env)
needs = env.needs_all_needs.values()
needs_per_id_build_path = self.app.config.needs_per_id_build_path
needs_build_json_per_id_path = self.app.config.needs_build_json_per_id_path
filter_string = self.app.config.needs_builder_filter
filtered_needs = filter_needs(self.app, needs, filter_string)
needs_id_json_dir = os.path.join(self.outdir, needs_per_id_build_path)
needs_id_json_dir = os.path.join(self.outdir, needs_build_json_per_id_path)
if not os.path.exists(needs_id_json_dir):
os.mkdir(needs_id_json_dir)
for need in filtered_needs:
Expand All @@ -187,7 +189,7 @@ def finish(self) -> None:
with open(fname, "w") as f:
json.dump(needs_id_dict, f, indent=4)
except Exception as e:
danwos marked this conversation as resolved.
Show resolved Hide resolved
log.error(f"Error during writing json file: {e}_{id}")
log.error(f"Needs-ID Builder {id} error: {e}")
log.info("Needs_id successfully exported")

def get_outdated_docs(self) -> Iterable[str]:
Expand All @@ -209,7 +211,7 @@ def get_target_uri(self, _docname: str, _typ: Optional[str] = None) -> str:
def build_needs_id_json(app: Sphinx, _exception: Exception) -> None:
env = unwrap(app.env)

if not env.config.needs_per_id:
if not env.config.needs_build_json_per_id:
return

# Do not create an additional needs_json for every needs_id, if builder is already "needs_id".
Expand Down
4 changes: 2 additions & 2 deletions sphinx_needs/needs.py
Original file line number Diff line number Diff line change
Expand Up @@ -283,8 +283,8 @@ def setup(app: Sphinx) -> Dict[str, Any]:
app.add_config_value("needs_debug_measurement", False, "html", types=[dict])

# add json file per needs_id
app.add_config_value("needs_per_id", False, "html", types=[bool])
app.add_config_value("needs_per_id_build_path", "needs_id", "html")
app.add_config_value("needs_build_json_per_id", False, "html", types=[bool])
app.add_config_value("needs_build_json_per_id_path", "needs_id", "html")
# Define nodes
app.add_node(Need, html=(html_visit, html_depart), latex=(latex_visit, latex_depart))
app.add_node(
Expand Down
21 changes: 21 additions & 0 deletions tests/doc_test/docs_needs_id_builder/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Minimal makefile for Sphinx documentation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file is not necessary for the test

#

# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = needstestdocs
SOURCEDIR = .
BUILDDIR = _build
ERRORLOG = error.log

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" -W --keep-going -w $(ERRORLOG) $(SPHINXOPTS) $(O)
162 changes: 162 additions & 0 deletions tests/doc_test/docs_needs_id_builder/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
#
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file can be a lot more minimal; only include what is absolutely necessary to pass the tests

# needs test docs documentation build configuration file, created by
# sphinx-quickstart on Tue Mar 28 11:37:14 2017.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys

sys.path.insert(0, os.path.abspath("../../sphinxcontrib"))

# -- General configuration ------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.

extensions = ["sphinx_needs"]

needs_table_style = "TABLE"


needs_types = [
{"directive": "story", "title": "User Story", "prefix": "US_", "color": "#BFD8D2", "style": "node"},
{"directive": "spec", "title": "Specification", "prefix": "SP_", "color": "#FEDCD2", "style": "node"},
{"directive": "impl", "title": "Implementation", "prefix": "IM_", "color": "#DF744A", "style": "node"},
{"directive": "test", "title": "Test Case", "prefix": "TC_", "color": "#DCB239", "style": "node"},
]

plantuml = "java -jar %s" % os.path.join(os.path.dirname(__file__), "..", "utils", "plantuml.jar")
plantuml_output_format = "svg"

needs_file = "custom_needs_test.json"

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = ".rst"

# The master toctree document.
master_doc = "index"

# General information about the project.
project = "needs test docs"
copyright = "2017, team useblocks"
author = "team useblocks"

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = "1.0"
# The full version, including alpha/beta/rc tags.
release = "1.0"

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = "en"

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"

# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False

# -- Options for HTML output ----------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "alabaster"

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ["_static"]

# -- Options for HTMLHelp output ------------------------------------------

# Output file base name for HTML help builder.
htmlhelp_basename = "needstestdocsdoc"

# -- Options for LaTeX output ---------------------------------------------

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}

# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, "needstestdocs.tex", "needs test docs Documentation", "team useblocks", "manual"),
]

# -- Options for manual page output ---------------------------------------

# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [(master_doc, "needstestdocs", "needs test docs Documentation", [author], 1)]

# -- Options for Texinfo output -------------------------------------------

# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(
master_doc,
"needstestdocs",
"needs test docs Documentation",
author,
"needstestdocs",
"One line description of project.",
"Miscellaneous",
),
]
14 changes: 14 additions & 0 deletions tests/doc_test/docs_needs_id_builder/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
TEST DOCUMENT NEEDS Builder
===========================

.. story:: A story
:status: in progress
:tags: 1

.. test:: Test example
:id: TC_001
:status: open

.. test:: Negative test example
:id: TC_NEG_001
:status: closed
Loading