Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add picasso commands in simple way #2

Merged
merged 25 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
3882d9b
feat: add picasso commands in simple way
MaferMazu Jul 31, 2024
77e973a
test: add workflows and fix tests
MaferMazu Aug 2, 2024
dea0046
docs: add information in the readme
MaferMazu Aug 2, 2024
c740aa4
test: fix tests
MaferMazu Aug 2, 2024
a4c769b
fix: add importlib_resources for palm support
MaferMazu Aug 5, 2024
9e035ec
fix: add commands in picasso installation
MaferMazu Aug 5, 2024
f977bd3
feat: add backward compatibility
MaferMazu Aug 16, 2024
1a5d0b5
fix: fix tests
MaferMazu Aug 16, 2024
1cd0557
fix: add condition before MOUNTED_DIRECTORIES
MaferMazu Aug 16, 2024
e48b36d
test: add integration tests
MaferMazu Aug 16, 2024
9ca43a9
fix: implement feedback improvements
MaferMazu Aug 20, 2024
d1110cb
fix: correct test
MaferMazu Aug 20, 2024
e55f4b9
docs: improve usage description
MaferMazu Aug 20, 2024
a078a8b
fix: implement feedback
MaferMazu Aug 20, 2024
3c8dc2e
test: add again the test-types
MaferMazu Aug 20, 2024
e1ad584
fix: apply feedback about quick checks and separate methods
MaferMazu Aug 22, 2024
0203763
docs: update the readme format
MaferMazu Aug 23, 2024
18f983e
chore: improve integration tests
MaferMazu Aug 23, 2024
c8eefa0
refactor: use handler to manage requirement tasks depending on tutor …
mariajgrimaldi Aug 22, 2024
5d8dc5b
refactor: improve the naming in enable_private_packages
MaferMazu Aug 26, 2024
821c5fc
docs: improve code docs
MaferMazu Aug 26, 2024
018b455
refactor: return all command error messages instead
mariajgrimaldi Aug 22, 2024
f03da32
fix: check the commands inside the loop and format
MaferMazu Aug 26, 2024
26fb79a
docs: add documentation about the type ignore
MaferMazu Aug 26, 2024
035f49d
fix: use typing Pattern instead of regex one
MaferMazu Aug 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/commitlint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: Lint Commit Messages
on: [pull_request]

jobs:
commitlint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: wagoid/commitlint-github-action@v6
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Run tests

on:
pull_request:
branches: [master]
branches: [main]

jobs:
tests:
Expand Down
115 changes: 112 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,16 @@
picasso plugin for `Tutor <https://docs.tutor.edly.io>`__
#########################################################
Picasso `Tutor`_ Plugin
#########################

picasso plugin for Tutor
|Maintainance Badge| |Test Badge|

.. |Maintainance Badge| image:: https://img.shields.io/badge/Status-Maintained-brightgreen
:alt: Maintainance Status
.. |Test Badge| image:: https://img.shields.io/github/actions/workflow/status/edunext/tutor-contrib-picasso/.github%2Fworkflows%2Ftests.yml?label=Test
:alt: GitHub Actions Workflow Test Status

Picasso is a `Tutor`_ plugin that simplifies pre-build processes, such as preparing Docker images with private requirements, executing commands before the build, and adding themes. Everything is managed through commands, making it easy to integrate into automated environments.

MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
This plugin is based on https://github.com/eduNEXT/tutor-contrib-edunext-distro


Installation
Expand All @@ -16,10 +25,110 @@ Usage

.. code-block:: bash

# To enable the plugin
tutor plugins enable picasso

# Show help
tutor picasso -h

# Enable themes
tutor picasso enable-themes

# Enable private packages
tutor picasso enable-private-packages

# Run Tutor commands
tutor picasso run-extra-commands

.. note::

Remember to run these commands before build your images if they are needed.

MaferMazu marked this conversation as resolved.
Show resolved Hide resolved

Compatibility notes
*******************

This plugin was tested from Palm release.

About the commands
*******************
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved

Enable Private Packages
^^^^^^^^^^^^^^^^^^^^^^^^

This command allows the installation of private Open edX Django apps. It clones the private repository and, through the ``tutor mounts`` command, adds it to the Dockerfile for inclusion in the build process. The input it takes is:

MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
.. code-block:: yaml
PICASSO_<YOUR_PACKAGE_NAME>_DPKG:
name: <your_package_name>
repo: <your SSH URL for cloning the repo. e.g., [email protected]:yourorg/package.git>
version: <your branch, tag o release for cloning. e.g., v5.2.0>

MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
.. note::

It is needed to use the SSH URL to clone private packages.

.. warning::

For the mount to work correctly and include the package in the Dockerfile, it must be added to a tutor filter ``MOUNTED_DIRECTORIES``. By default, Picasso adds ``eox-*`` packages. If you need to add another private package, don't forget to include this configuration in a Tutor plugin.

.. code-block:: python

hooks.Filters.MOUNTED_DIRECTORIES.add_items(
[
("openedx", "<your_package_name>"),
]
)


.. note::

If you want to use public packages, we recommend using the ``OPEN_EDX_EXTRA_PIP_REQUIREMENTS`` variable in the ``config.yml`` of your Tutor environment.


Enable Themes
^^^^^^^^^^^^^^

This command clones your theme repository into the folder that Tutor uses for themes. Documentation available at `Installing custom theme`_ tutorial. The input it takes is:

.. code-block:: yaml
PICASSO_THEMES:
- name: <your_theme_name>
repo: <your SSH URL for cloning the repo. e.g., [email protected]:yourorg/theme.git>
version: <your branch, tag o release for cloning. e.g., edunext/redwood.master>
- name: <another_theme_name>
repo: <your SSH URL for cloning the repo. e.g., [email protected]:yourorg/another_theme.git>
version: <your branch, tag o release for cloning. e.g., edunext/redwood.blue>

MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
.. note::

If your theme repository is public, you can also use the HTTPS URL in ``repo``.

.. note::

Don't forget to add extra configurations in a Tutor plugin if your theme requires it.


Run Extra Commands
^^^^^^^^^^^^^^^^^^^

This command allows you to run a list of Tutor commands. These commands are executed in bash and, for security reasons, are restricted to running only Tutor commands. The input it takes is:

.. code-block:: yaml
PICASSO_EXTRA_COMMANDS:
- <A tutor command. e.g., tutor plugins index add X>
- <A tutor command. e.g., tutor plugins install mfe>
- <A tutor command. e.g., tutor picasso enable-themes>
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
- <A tutor command. e.g., tutor config save>
.
.
.

License
*******

This software is licensed under the terms of the AGPLv3.


.. _Tutor: https://docs.tutor.edly.io
.. _Installing custom theme: https://docs.tutor.edly.io/tutorials/theming.html#theming
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@ def load_about():
packages=find_packages(exclude=["tests*"]),
include_package_data=True,
python_requires=">=3.8",
install_requires=["tutor>=18.0.0,<19.0.0"],
install_requires=["tutor"],
extras_require={
"dev": [
"tutor[dev]>=18.0.0,<19.0.0",
"tutor[dev]",
]
},
entry_points={
Expand Down
2 changes: 1 addition & 1 deletion tutorpicasso/__about__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "18.0.0"
__version__ = "0.1.0"
23 changes: 23 additions & 0 deletions tutorpicasso/commands/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""
Picasso commands group.
"""

import click

from tutorpicasso.commands.enable_private_packages import enable_private_packages
from tutorpicasso.commands.enable_themes import enable_themes
from tutorpicasso.commands.run_extra_commands import run_extra_commands


@click.group(help="Run picasso commands")
def picasso() -> None:
"""
Main picasso command group.

This command group provides functionality to run picasso commands.
"""


picasso.add_command(enable_private_packages)
picasso.add_command(run_extra_commands)
picasso.add_command(enable_themes)
64 changes: 64 additions & 0 deletions tutorpicasso/commands/enable_private_packages.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import os
import subprocess

# Was necessary to use this for compatibility with Python 3.8
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
from typing import Any, Dict

import click
from tutor import config as tutor_config


@click.command(name="enable-private-packages", help="Enable picasso private packages")
def enable_private_packages() -> None:
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
"""
Enable private packages command.

This command enables picasso private packages by cloning the packages and
defining them as private.

Raises:
Exception: If an error occurs during the cloning or defining process.
"""
tutor_root = (
subprocess.check_output("tutor config printroot", shell=True)
.decode("utf-8")
.strip()
)
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
config = tutor_config.load(tutor_root)
packages = get_picasso_packages(config)
for package, info in packages.items():
try:
if not {"name", "repo", "version"}.issubset(info):
raise KeyError(
f"{package} is missing one of the required keys: 'name', 'repo', 'version'"
)
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved

if os.path.isdir(f'{tutor_root}/{info["name"]}'):
subprocess.call(["rm", "-rf", f'{tutor_root}/{info["name"]}'])

subprocess.call(
["git", "clone", "-b", info["version"], info["repo"]], cwd=tutor_root
)
subprocess.call(["tutor", "mounts", "add", f'{tutor_root}/{info["name"]}'])

except KeyError as e:
raise click.ClickException(str(e))


def get_picasso_packages(settings: Dict[str, Any]) -> Dict[str, Dict[str, Any]]:
"""
Get the distribution packages from the provided settings.

Args:
settings (dict): The tutor configuration settings.

Returns:
dict: A dictionary of distribution packages, where the keys are package names
and the values are package details.
"""
picasso_packages = {
key: val
for key, val in settings.items()
if key.endswith("_DPKG") and val != "None"
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
}
return picasso_packages
41 changes: 41 additions & 0 deletions tutorpicasso/commands/enable_themes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import os
import subprocess
from typing import Any

import click
from tutor import config as tutor_config


@click.command(name="enable-themes", help="Enable picasso themes")
def enable_themes() -> None:
"""
Enable picasso themes.

This function enables the themes specified in the `PICASSO_THEMES` configuration
and applies them using the ThemeEnabler and ThemeGitRepository classes.
"""
tutor_root = (
subprocess.check_output("tutor config printroot", shell=True)
.decode("utf-8")
.strip()
)
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
config: Any = tutor_config.load(tutor_root)

if config.get("PICASSO_THEMES"):
for theme in config["PICASSO_THEMES"]:
try:
if not {"name", "repo", "version"}.issubset(theme.keys()):
raise KeyError(
f"{theme} is missing one or more required keys: "
"'name', 'repo', 'version'"
)

theme_path = f'{tutor_root}/env/build/openedx/themes/{theme["name"]}'
if os.path.isdir(theme_path):
subprocess.call(["rm", "-rf", theme_path])

subprocess.call(
["git", "clone", "-b", theme["version"], theme["repo"], theme_path],
)
except KeyError as e:
raise click.ClickException(f"Error: {str(e)}")
MaferMazu marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading