Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with using oci_load and external dependencies #752

Open
theunwisewolf opened this issue Dec 13, 2024 · 1 comment
Open

Issue with using oci_load and external dependencies #752

theunwisewolf opened this issue Dec 13, 2024 · 1 comment

Comments

@theunwisewolf
Copy link

theunwisewolf commented Dec 13, 2024

I have 2 bazel modules. One at the root and the other in a sub directory. Example:

root
  /libA
    /BUILD.bazel <-- uses oci_load()
    /MODULE.bazel
  MODULE.bazel <-- Imports libA's module

In the root MODULE.bazel I have:

bazel_dep(name = "libA", version = "1.0")
local_path_override(module_name = "libA", path = "./libA")

And libA's BUILD file has:

... build app, package to tar, transition binary

oci_load(
    name = "load",
    image = ":transitioned_image",
    repo_tags = "tags.txt",
)

When I run bazel run @libA//:load, the command fails with the error:

tar: Error reading archive -: Can't open external/libA+/image/blobs/sha256/d057aa77ea6c6e060f84c1bbe1a261478509b8d0f11bdbbd3ca6a5a3bbd1e2d0
tar: Error exit delayed from previous errors.
open /var/lib/docker/tmp/docker-import-1307345443/repositories: no such file or directory

If I run bazel run //:load inside libA dir, it runs fine. I can also run the app from the root directory without any issues. It's only the image creation that fails.
I suspect the '+' symbol in the name is causing some problems for the tar command.

So the oci_load() does not work when used with external dependencies. Any workarounds/fixes?

@sfc-gh-mhazy
Copy link
Contributor

sfc-gh-mhazy commented Dec 19, 2024

I am running into the same issues, I think the problem is with load implementation and path remapping.

I believe in

"{{manifest_root}}": manifest_json.root.path,
: manifest_root, image_root and workspace_name are not working with running oci_load defined in different module / repository.

Suppose we have a workspace / module named "main" and it imports "other" that has @other//path:my_oci_load

The following are the problems:

  1. ctx.workspace_name seems to always returns the name of "main" workspace. But we would want "other"
  2. file.root.path returns the root of all outputs - lets say "/path/to/my/root"
    • But whole path of file from @other//path:target is /path/to/my/root/external/other/path/target. For targets in "main" it would be /path/to/my/root/main/path/target (No external prefix)

In the end, the load executable would not strip enough from the path and then go into wrong directory ($RUNFILES_DIR/main, while it should go to $RUNFILES_DIR/other)

I am not sure what is the best solution here, but I generally tend to use file.short_path to distinguish between these two cases (if short path starts with .. -> this is external repository and next directory is a workspace name. Not sure if this is the "clean" solution though.

Roughly:

def _get_workspace(ctx, file):
    if file.short_path.startswith("../"):
        short_path = file.short_path
        split = short_path.split("/")
        return split[1]
    return ctx.workspace_name

def _get_prefix_to_trim(ctx, file):
    workspace = _get_workspace(ctx, file)
    if workspace != ctx.workspace_name:
        return file.root.path + "/external/" + workspace
    return file.root.path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants