Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

venv dependency not found when running via caldera-atlas #1

Open
lilyjw opened this issue Mar 15, 2023 · 2 comments
Open

venv dependency not found when running via caldera-atlas #1

lilyjw opened this issue Mar 15, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@lilyjw
Copy link
Contributor

lilyjw commented Mar 15, 2023

Describe the bug
Arsenal encounters an error about not having python3.10-venv available during start within caldera-atlas usage, i.e. CALDERA in a Docker container.

Not sure where to address - is there a way to define additional dependencies for CALDERA to install from a plugin? Or is this something caldera-atlas should do.

To Reproduce
Steps to reproduce the behavior:

Set up caldera-atlas

  1. git clone --recursive https://github.com/mitre-atlas/caldera-atlas
    If one has SSH keys uploaded, this should check out all submodules.
    Otherwise, there may be failures because some submodules have SSH URLs. To resolve:
    a. git checkout https to get the branch with HTTPS URLs
    b. git submodule sync --recursive && git submodule update --init

  2. docker-compose build
    If one gets an OpenSSL UnsafeLegacyRenegotiation error, resolve it as one sees fit but can add this to caldera-atlas/caldera/Dockerfile ~L18 after the RUN apt-get ...

# OpenSSL 3.0 disables UnsafeLegacyRenegotiation by default, must re-enable it for some endpoints (see https://github.com/dotnet/runtime/issues/80641)
RUN sed -i 's/providers = provider_sect/providers = provider_sect\n\
        ssl_conf = ssl_sect\n\
        \n\
        [ssl_sect]\n\
        system_default = system_default_sect\n\
        \n\
        [system_default_sect]\n\
        Options = UnsafeLegacyRenegotiation/' /etc/ssl/openssl.cnf
  1. docker-compose up -d

The container exits shortly after this - see the trace below

Expected behavior
Container stays up

Trace
From the Docker container log of CALDERA (the caldera-atlas service):

2023-03-15 16:37:41 - INFO  (app_svc.py:116 load) Enabled plugin: access
2023-03-15 16:37:41 - DEBUG (arsenal_svc.py:122 _create_venv)
Building cf_venv...
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/root/venv/cf_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

2023-03-15 16:37:41 - ERROR (base_events.py:1744 default_exception_handler) Task exception was never retrieved
future: <Task finished name='Task-33' coro=<AppService.load_plugins.<locals>.load() done, defined at /usr/src/app/app/service/app_svc.py:108> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/usr/lib/python3.10/venv/__init__.py", line 318, in _setup_pip
    subprocess.check_output(
  File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/venv/cf_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/base_events.py", line 633, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 600, in run_forever
    self._run_once()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1896, in _run_once
    handle._run()
  File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/app/app/service/app_svc.py", line 115, in load
    await plugin.enable(self.get_services())
  File "/usr/src/app/app/objects/c_plugin.py", line 67, in enable
    await plugin(services)
  File "/usr/src/app/plugins/arsenal/hook.py", line 15, in enable
    await arsenal_svc.populate_venv(env_name='cf_venv', with_counterfit=True)
  File "/usr/src/app/plugins/arsenal/app/arsenal_svc.py", line 62, in populate_venv
    await self._create_venv(env_name=env_name)
  File "/usr/src/app/plugins/arsenal/app/arsenal_svc.py", line 136, in _create_venv
    builder.create(env_dir=env_dir)
  File "/usr/lib/python3.10/venv/__init__.py", line 75, in create
    self._setup_pip(context)
  File "/usr/lib/python3.10/venv/__init__.py", line 336, in _setup_pip
    sys.exit(1)
SystemExit: 1

Desktop (please complete the following information):
Ubuntu 22.04 but running CALDERA as a container using its own Dockerfile, which uses ubuntu:latest

Additional context
Add any other context about the problem here.

@lilyjw lilyjw added the bug Something isn't working label Mar 15, 2023
@mdotter-mitre
Copy link
Contributor

mdotter-mitre commented Mar 30, 2023

@lilyjw Core plugins to caldera have dependencies installed via a requirements file during the build of the docker image - as this is a stand-alone plugin at this time, we need to build the Dockerfile with these dependencies added.

Thank you for identifying this bug. I'll create a PR to update the repo https://github.com/mitre-atlas/caldera-atlas with python3.10-venv included!

@lilyjw
Copy link
Contributor Author

lilyjw commented May 4, 2023

Temporary workaround: Add python3-venv to the RUN apt-get ... step in the caldera/Dockerfile, then re-run docker-compose build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants