feat: Add support for pipeline plugins

[MoritzWeber0]

Our Systems engineering toolchain team offers several model modifiers and model derivative jobs.

Example for model modifiers are:

• ID injection bot (inject IDs into model elements)

Example for model derivates:

• Model complexity badge generation
• Diagram cache generation
• Automatic documentation generation

Synchronisation between TeamForCapella and Git is also considered as job, somewhere in between.

Currently, all "jobs" run in the Gitlab CI. There is no UI for the setup of these jobs. Each job repository provides a Gitlab CI templates, which can be easily integrated in project repositories. However, this approach is not intuitive for users / system architects, leading to operational support effort for operation teams, which have to take care of the creation and updates of jobs.

Another disadavantage of the current approach is the Gitlab dependency. Not all environments have a Gitlab instance available. Getting it running on other platforms maintaining templates for different CI/CD platforms is a lot of effort.

This commit adds a plugin/job store to the Capella Colloration Manager. A plugin/job is still developed in a Git repository. Plugins have to define a mbse-works-plugin.yml, providing metadata, input, output, trigger and job information.

With this information, registered plugins can be easily integrated into projects. The integration should be intuitive for project leads.

The steps are:

• Create a new pipeline
• Select a supported plugin from the store
• Configure the plugin: Supported types for the beginning are:
  • git (Select a linked git model from the project, so that the job can access the Git repository information)
  • t4c (Select a linked T4C repository/project from the project). The job gets the connection information + a session token.
  • yml (Most flexible option, the yml configuration file is mounted into the job container, validation via json schema)
  • environment (Key/value pairs, which are used as environment variable for the job)
• Confirm the creation

As part of this commit, there are only two options:

• Run the pipeline manually
• Run it as 3am during the night.

Currently, there is no locking and pipeline/job dependency support. Triggering pipelines via webhooks is planned as this would provide a integration with most Git repository servers.

TODO:

• Preview pipeline creation, create pipelines properly in the backend.
• Trigger a pipeline (resolve all required variables and configuration).
• Create CronJob if cron is used as trigger target.
• Write a YAML schema for the plugin file.

[codecov]

Codecov Report

Attention: 490 lines in your changes are missing coverage. Please review.

Comparison is base (5dbbe56) 72.50% compared to head (873ec8c) 65.73%.

❗ Current head 873ec8c differs from pull request most recent head 576fb32. Consider uploading reports for the commit 576fb32 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #722      +/-   ##
==========================================
- Coverage   72.50%   65.73%   -6.78%     
==========================================
  Files         162      178      +16     
  Lines        5289     5633     +344     
  Branches      592      604      +12     
==========================================
- Hits         3835     3703     -132     
- Misses       1322     1826     +504     
+ Partials      132      104      -28     

Files	Coverage Δ
backend/capellacollab/core/database/__init__.py	70.83% <100.00%> (ø)
backend/capellacollab/core/database/models.py	100.00% <100.00%> (ø)
backend/capellacollab/plugins/models.py	100.00% <100.00%> (ø)
backend/capellacollab/routes.py	100.00% <100.00%> (ø)
backend/capellacollab/plugins/injectables.py	62.50% <62.50%> (ø)
...ollab/projects/toolmodels/pipelines/runs/helper.py	0.00% <0.00%> (ø)
...collab/projects/toolmodels/pipelines/validation.py	0.00% <0.00%> (ø)
...ollab/projects/toolmodels/pipelines/injectables.py	0.00% <0.00%> (ø)
backend/capellacollab/plugins/schema.py	0.00% <0.00%> (ø)
...collab/projects/toolmodels/pipelines/exceptions.py	0.00% <0.00%> (ø)
... and 11 more

... and 15 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sonarcloud]

SonarCloud Quality Gate failed.   

2 Bugs
0 Vulnerabilities
1 Security Hotspot
11 Code Smells

No Coverage information
0.0% Duplication

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

[MoritzWeber0]

Due to the number of conflicts and the state of the PR, I'll close it. It can still be used as inspiration for future implementations.