Added documentation website (#1206)

.github/workflows/build.yml

@@ -7,8 +7,12 @@ on:
       - main
     tags:
       - v*.**
+    paths:
+      - "!docs/**"
   pull_request:
     types: [opened, synchronize, reopened]
+    paths:
+      - "!docs/**"
 
 jobs:
   build-cpgo-osx:

.github/workflows/docs.yml

@@ -0,0 +1,46 @@
+name: docs
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - docs/**
+  pull_request:
+    types: [opened, synchronize, reopened]
+    paths:
+      - docs/**
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    if: github.event.repository.fork == false
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Install python3
+        uses: actions/setup-python@v3
+        with:
+          python-version: 3.x
+      - name: Cache
+        uses: actions/cache@v3
+        with:
+          key: ${{ github.ref }}
+          path: .cache
+      - name: Install Material for MkDocs
+        run: |
+          pip install mkdocs-material pillow cairosvg
+          pip install -r docs/mkdocs-material-plugins.txt
+      - name: Build
+        run: cd docs && mkdocs build --clean --config-file mkdocs.yaml -d site -v
+      - name: Publish main
+        if: github.ref == 'refs/heads/main'
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          folder: docs/site
+      - name: Publish version
+        if: startsWith(github.ref, 'refs/tags/v') && !contains(github.ref, 'beta') && !contains(github.ref, 'alpha')
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          folder: docs/site

docs/Dockerfile

@@ -0,0 +1,13 @@
+FROM squidfunk/mkdocs-material
+
+# Ensure installation of required packages
+# - uses updated cache on-the-fly without storing it locally
+RUN apk --no-cache add \
+    git
+
+# Install additional plugins (cf. `./mkdocs-material-plugins.txt`)
+COPY mkdocs-material-plugins.txt /
+RUN python -m pip install --no-cache-dir -r /mkdocs-material-plugins.txt
+
+# Trust git directory for git revision plugin
+RUN git config --global --add safe.directory /docs

docs/README.md

@@ -0,0 +1,17 @@
+# CPG Documentation
+
+The documentation for CPG is built with [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/) and hosted with GitHub Pages.
+
+Simply use the provided [Dockerfiles](./Dockerfile) in this directory. 
+It includes all the necessary plugins. 
+
+To build the Docker image use:
+```shell
+docker build -t mkdocs-material .
+```
+Afterwards, you can start a local development server:
+```shell
+docker run --rm -it -p 8000:8000 -v ${PWD}:/docs mkdocs-material
+```
+
+Please note, that the `git-revision-date-localized` plugin does not work with git worktrees.

docs/docs/CPG/impl/index.md

@@ -0,0 +1,26 @@
+---
+title: "Implementation and Concepts"
+linkTitle: "Implementation and Concepts"
+weight: 20
+no_list: false
+menu:
+  main:
+    weight: 20
+description: >
+    The CPG library is a language-agnostic graph representation of source code.
+---
+
+# Implementation and Concepts
+
+The translation of source code to the graph consists of two main steps. First,
+the source code is parsed and transferred to the CPG nodes by a so-called
+**Language Frontend**. Then, **Passes** refine the information which is kept in
+the graph. These two stages are strictly separated one from each other.
+
+![Overview of the CPG pipeline](../../assets/img/cpg-flow.png) 
+{ align=center }
+
+
+* [Languages and Language Frontends](./language)
+* [Scopes](./scopes)
+* [Passes](./passes)

docs/docs/CPG/impl/language.md

@@ -0,0 +1,100 @@
+---
+title: "Implementation and Concepts - Language Frontends"
+linkTitle: "Implementation and Concepts - Language Frontends"
+weight: 20
+no_list: false
+menu:
+  main:
+    weight: 20
+description: >
+    The CPG library is a language-agnostic graph representation of source code.
+---
+
+
+
+# Implementation and Concepts: Language and Language Frontends
+
+Even though we are aiming for a language-independent representation of source
+code, we still need to parse source code depending on the original programming
+language used. Therefore, we are introduce two concepts that help developers and
+users to understand how the CPG translates language-specific code into an
+abstract form. 
+
+## `Language`
+
+The first concept is a `Language`. It represents the programming language as a
+general concept and contains meta-information about it. This includes:
+
+* The name of the language, e.g. C++
+* The delimiter used to separate namespaces, e.g., `::`
+* The [`LanguageFrontend`](#LanguageFrontend) used to parse it
+* Additional [`LanguageTrait`](#LanguageTrait) implementations
+
+Each `Node` has a `language` property that specifies its language.
+
+### `LanguageTrait`
+
+A language trait aims to further categorize a programming language based on
+conceptual paradigms. This can be easily extended by introducing new interfaces
+based on `LanguageTrait`. Examples include:
+
+* Are default arguments supported?
+* Does the language have structs or classes?
+* Are function pointers supported?
+* Are templates or generics used in the language?
+* Do we need some special knowledge to resolve symbols, calls, variables?
+
+These traits are used during the pass execution phase to fine-tune things like
+call resolution or type hierarchies.
+
+## `LanguageFrontend`
+
+In contrast to the `Language` concept, which represents the generic concept of a
+programming language, a `LanguageFrontend` is a specific module in the CPG
+library that does the actual translating of a programming language's source code
+into our CPG representation.
+
+At minimum a language frontend needs to parse the languages' code and translate
+it to specific CPG nodes. It will probably use some library to retrieve the
+abstract syntax tree (AST). The frontend will set the nodes' `AST` edges and
+establish proper scopes via the scope manager. Everything else, such as call or
+symbol resolving is optional and will be done by later passes. However, if a
+language frontend is confident in setting edges, such as `REFERS_TO`, it is
+allowed to and this is respected by later passes. However, one must be extremely
+careful in doing so.
+
+The frontend has a limited life-cycle and only exists during the *translation*
+phase. Later, during the execution of passes, the language frontend will not
+exist anymore. Language-specific customization of passes are done using
+[`LanguageTraits`](#LanguageTrait).
+
+To create nodes, a language frontend MUST use the node builder functions in the
+`ExpressionBuilder`, `DeclarationBuilder` or `StatementBuilder`. These are
+Kotlin extension functions that automatically inject the context, such as
+language, scope or code location of a language frontend or its handler into the
+created nodes.
+
+## Supporting a new language
+
+To support a new language, all you have to do is to
+
+* Provide a new `Language`. Here, you have to think about the features of the
+  programming language and which `LanguageTraits` the respective language has to
+  implement. With this, you provide the respective fine-tuning to make the
+  Passes work properly.
+* Implement a new `LanguageFrontend` which is executed if a file matches the new
+  `Language`. The requirements of the frontends are described above.
+
+To make use of the respective frontend by the CPG, you have to configure the
+translation accordingly. This is done by the `TranslationConfiguration` where
+you register your new language by calling one of the `registerLanguage()`
+methods. As an example
+```kotlin
+val config: TranslationConfiguration = TranslationConfiguration
+    .builder()
+    // More configuration
+    .registerLanguage(MyNewLanguage()) // Option 1
+    .registerLanguage<MyOtherNewLanguage>() // Option 2
+    .registerLanguage("MyThirdNewLanguage") // Option 3
+    .build()
+```

docs/docs/CPG/impl/passes.md

@@ -0,0 +1,69 @@
+---
+title: "Implementation and Concepts - Passes"
+linkTitle: "Implementation and Concepts - Passes"
+weight: 20
+no_list: false
+menu:
+  main:
+    weight: 20
+description: >
+    The CPG library is a language-agnostic graph representation of source code.
+---
+
+
+
+# Implementation and Concepts: Passes
+
+## What is a Pass?
+
+Passes get a prebuilt CPG that at least contains the CPG-AST and output a
+modified graph. Their purpose is to extend the syntactic representation of code
+with additional nodes and edges to represent the semantics of the program.
+Passes can be executed in sequence, where the output of the previous pass serves
+as input of the next pass.
+
+## Creating a new Pass
+
+The user of the cpg library can implement her own passes. Each pass needs to
+extend the class `Pass` and implement its base function`accept(result: TranslationResult)`.
+The remaining structure of the pass is free to be designed by the
+implementer.
+
+## Registering a Pass
+
+A newly created pass has to be registered with the `TranslationManager` through
+its builder by calling
+```
+val configuration = TranslationConfiguration.builder().
+    // ...
+    .registerPass(...)
+```
+## Modifying a Pass
+
+A preexisting pass can be modified by extending it and overwriting its
+functions. For this purpose, all member functions of existing library passes
+have the visibility `protected`. Depending on the modified pass, internal
+constructs have to be respected.
+
+For example, the `EvaluationOrderGraphPass` uses an internal handle structure.
+When extending this pass, it is necessary to add handlers of new Node types to
+the internal handler map. If a developer needs to override an exisiting handler,
+the handle has to be implemented with the same signature to use the polymorphism
+feature. Additionally the mapping of `node type -> handler` needs to be replaced
+by a new entry `node type -> overridden` handler.
+
+## Ordering Passes
+Passes may depend on the information added by another pass. This requires us to
+enforce the order in which passes are executed. To do so, we provide the
+following annotations for the passes:
+
+* `DependsOn(other: KClass<out Pass>, softDependency: Boolean = false)` -- The annotated pass is executed after
+   the other pass(es). If `softDependency` is set to `false`, it automatically
+   registers these passes if they haven't been registered by the user.
+* `ExecuteBefore(other: KClass<out Pass>, ...)` -- The annotated pass is executed
+   before the other pass(es) specified.
+* `ExecuteFirst` -- The annotated pass is executed as the first pass if possible.
+* `ExecuteLast` -- The annotated pass is executed as the last pass if possible.
+* `RequiredFrontend(frontend: KClass<out LanguageFrontend>)` -- The annotated pass
+   is only executed if the frontend has been used.
+

docs/docs/CPG/impl/scopes.md

@@ -0,0 +1,15 @@
+---
+title: "Implementation and Concepts - Scopes"
+linkTitle: "Implementation and Concepts - Scopes"
+weight: 20
+no_list: false
+menu:
+  main:
+    weight: 20
+description: >
+    The CPG library is a language-agnostic graph representation of source code.
+---
+
+
+# Implementation and Concepts: Scopes and Scope Manger
+

docs/docs/CPG/index.md

@@ -0,0 +1,14 @@
+---
+title: "Documentation"
+linkTitle: "Documentation"
+weight: 20
+no_list: true
+menu:
+  main:
+    weight: 20
+description: >
+    The CPG library is a language-agnostic graph representation of source code.
+---
+
+
+