Gazelle unable to resolve external Go library from go.mod. Manually adding to use_repo list works around it, but bazel mod tidy removes it.

[jasonbdly]

What version of gazelle are you using?

0.40.0

What version of rules_go are you using?

0.50.1

What version of Bazel are you using?

Bazelisk version: 1.24.0
Bazel version: 7.4.1

Does this issue reproduce with the latest releases of all the above?

Yes, I made sure to update everything mentioned above and retest. Unfortunately I'm still running into the same issue.

What operating system and processor architecture are you using?

Working within Ubuntu (amd64) on WSL2 on a Windows 11 host machine.

What did you do?

I recently added these two imports to my project:

• github.com/lestrrat-go/jwx/v3/jwt
• github.com/lestrrat-go/jwx/v3/jwa

To do this, I followed these steps:

1. Ran bazel run @rules_go//go -- get github.com/lestrrat-go/jwx/v3/jwt and bazel run @rules_go//go -- get github.com/lestrrat-go/jwx/v3/jwa.
2. Verified that my go.mod file was updated to include the underlying import github.com/lestrrat-go/jwx/v3.
3. Updated a go file to import these two packages.
4. Ran bazel run //:gazelle at the root of the project to update the associated BUILD file.
5. Ran bazel mod tidy --enable_bzlmod at the root of the project (didn't make any changes).
6. Attempted to run the target that includes a sub-step that builds the updated go file. In my case this involves building an image and loading it to my local image registry via oci_load from @rules_oci, but theoretically this would apply to a simple go build target as well.

What did you expect to see?

I expected to see that the target ran successfully.

What did you see instead?

The target failed with the following error message, seemingly unable to resolve the new external import:

$ bazel run //src/services/identity/exec:load_image --logging=6
ERROR: no such package '@@[unknown repo 'com_github_lestrrat_go_jwx_v3' requested from @@]//jwa': The repository '@@[unknown repo 'com_github_lestrrat_go_jwx_v3' requested from @@]' could not be resolved: No repository visible as '@com_github_lestrrat_go_jwx_v3' from main repository
ERROR: /home/jasonbdly/dev/go/curtain/src/services/identity/BUILD:3:11: no such package '@@[unknown repo 'com_github_lestrrat_go_jwx_v3' requested from @@]//jwa': The repository '@@[unknown repo 'com_github_lestrrat_go_jwx_v3' requested from @@]' could not be resolved: No repository visible as '@com_github_lestrrat_go_jwx_v3' from main repository and referenced by '//src/services/identity:identity'
ERROR: /home/jasonbdly/dev/go/curtain/src/services/identity/BUILD:3:11: no such package '@@[unknown repo 'com_github_lestrrat_go_jwx_v3' requested from @@]//jwt': The repository '@@[unknown repo 'com_github_lestrrat_go_jwx_v3' requested from @@]' could not be resolved: No repository visible as '@com_github_lestrrat_go_jwx_v3' from main repository and referenced by '//src/services/identity:identity'
ERROR: Analysis of target '//src/services/identity/exec:load_image' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.224s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: 
    Fetching repository @@aspect_bazel_lib~~toolchains~bsd_tar_linux_amd64; starting
    Fetching repository @@rules_oci~~oci~oci_regctl_linux_amd64; starting
    Fetching repository @@rules_oci~~oci~distroless_base; starting
    Fetching https://github.com/regclient/regclient/releases/download/v0.7.0/regctl-linux-amd64; Checking in SHA-256 cache
    Fetching https://github.com/aspect-build/bsdtar-prebuilt/releases/download/v3.7.4-4/tar_linux_amd64; Checking in SHA-256 cache

Out of curiosity, I tried running the aforementioned target again after manually adding "com_github_lestrrat_go_jwx_v3" to the list of dependencies passed to use_repo in my MODULE.bazel file and it ran successfully. Although it did print an error pointing out that an indirect dependency was in that list and I should run bazel mod tody to fix it. Running that command removes the explicit import I added, but as a result the original target fails to build again.

Reference - Contents of Relevant Files

MODULE.bazel

module(name = "curtain")

bazel_dep(name = "rules_go", version = "0.50.1")
bazel_dep(name = "rules_proto", version = "7.0.1")
bazel_dep(name = "aspect_bazel_lib", version = "2.9.4")
bazel_dep(name = "gazelle", version = "0.40.0")

go_sdk = use_extension("@rules_go//go:extensions.bzl", "go_sdk")
go_sdk.download(version = "1.22.9")

go_deps = use_extension("@gazelle//:extensions.bzl", "go_deps")
go_deps.from_file(go_mod = "//:go.mod")

# All *direct* Go dependencies of the module have to be listed explicitly.
use_repo(
    go_deps,
    "io_gorm_driver_mysql",
    "io_gorm_gorm",
    "org_golang_google_grpc",
    "org_golang_google_protobuf",
    "org_golang_x_crypto",
)

bazel_dep(name = "rules_pkg", version = "1.0.1")
bazel_dep(name = "rules_oci", version = "2.0.0")

oci = use_extension("@rules_oci//oci:extensions.bzl", "oci")
oci.pull(
    name = "distroless_base",
    image = "gcr.io/distroless/base",
    platforms = ["linux/amd64"],
    # 'latest' is not reproducible, but it's convenient.
    # During the build we print a WARNING message that includes recommended 'digest' and 'platforms'
    # values which you can use here in place of 'tag' to pin for reproducibility.
    tag = "latest",
)
use_repo(oci, "distroless_base", "distroless_base_linux_amd64")

//src/services/identity/BUILD

load("@rules_go//go:def.bzl", "go_library")

go_library(
    name = "identity",
    srcs = ["service.go"],
    importpath = "gitlab.com/jasonbdly/curtain/src/services/identity",
    visibility = [
        "//visibility:public",
    ],
    deps = [
        "//src/core/service",
        "//src/services/identity/models",
        "//src/services/identity/proto:identity_go_proto",
        "@com_github_lestrrat_go_jwx_v3//jwa",
        "@com_github_lestrrat_go_jwx_v3//jwt",
        "@io_gorm_gorm//:gorm",
        "@org_golang_google_grpc//:grpc",
        "@org_golang_x_crypto//bcrypt",
    ],
)

Snippet of imports from //src/services/identity/service.go

package identity

import (
	"context"
	b64 "encoding/base64"
	"errors"
	"log"
	"time"

	"github.com/lestrrat-go/jwx/v3/jwa"
	"github.com/lestrrat-go/jwx/v3/jwt"
	"golang.org/x/crypto/bcrypt"
	"google.golang.org/grpc"
	"gorm.io/gorm"

	"gitlab.com/jasonbdly/curtain/src/core/service"
	identity_models "gitlab.com/jasonbdly/curtain/src/services/identity/models"
	identity_pb "gitlab.com/jasonbdly/curtain/src/services/identity/proto"
)

My questions

Is there something special about this repository that requires an override? I'm not keenly aware of Gazelle's override options and where they might be used. I'd really appreciate some guidance on that front if possible!

The repository for the import I'm using is here: https://github.com/lestrrat-go/jwx. It appears that jwt and jwa are sub-folders rather than separate repositories, which explains the changes to my go.mod. The repository is bazel-compatible, but it is not using the newer Bzlmod format.

I'm also not entirely clear on the difference between direct and indirect dependencies as far as Gazelle is concerned. This dependency is explicitly listed in the BUILD file for the go file that uses it, so I assumed it would be considered a direct dependency of the project. However, bazel mod tidy seems to consider it indirect instead and I don't fully understand why that is. What's the meaning of direct vs indirect in that context?

[fmeum]

Could you share the content of your go.mod file? I don't see anything in that repo that would require special handling, so if you can provide me with something close to a standalone reproducer, I can probably figure out what's wrong in the setup and/or fix a bug in gazelle.

[jasonbdly]

Thanks for looking into this!
I set up a barebones reproduction of the issue in this repo: https://gitlab.com/jasonbdly/reproduction-of-gazelle-jwx-module-resolution-issue. The build target to run is in the root readme.

I had to make one additional change in MODULE.bazel to resolve an unrelated bug in rules_proto (bumped from 7.0.1 to 7.0.2), but the rest of the dependencies should be the same as in the original post.

[jasonbdly]

Hi @fmeum, were you able to reproduce the issue with that repo? Is there anything else I can provide to help debug this?

[jasonbdly]

I did a bit more digging into this and I think this comes down to this dependency being incorrectly marked as an indirect dependency (probably due to something I did incorrectly when adding the dependency, but I don't recall exactly what steps I used). This dependency is being used directly in my Go code, which I thought would be picked up by Gazelle so that it could deduce that it's a direct dependency rather than an indirect dependency, but that doesn't appear to be how that works.

I was able to mark that dependency as a direct dependency via go get and now bazel mod tidy adds the use_repo line itself. It also no longer complains about reproducibility when running the build target, so that seems like a step in the right direction. I'm not sure I fully understand how indirect vs direct dependencies are handled by bazel+gazelle in situations like this.

Since I'm using rules_go the full go get command was actually bazel run @rules_go//go -- get github.com/lestrrat-go/jwx/v3 (bazelisk is aliased as bazel in my case). After running this the // indirect comment marker was removed from that import in my go.mod file and "com_github_lestrrat_go_jwx_v3" was added to the use_repo call in MODULE.bazel.

Originally I think I must have attempted to "install" this dependency without the use of go get initially by just updating go.mod and running bazel mod tidy, which is what caused this issue. go mod tidy also isn't an option in my case due to the use of dynamically generated imports (internal proto targets for grpc services), which causes the command to hang at go: finding module for package path/to/service/proto.

[fmeum]

Gazelle gives you access to everything in go.mod that isn't marked as indirect. What is marked indirect is ultimately governed by go, but we do make it so that everything you @rules_go//go get explicitly is marked as a direct dependency to work around a usability issue in go tooling.

Ideally you would get go mod tidy to work in your project as that's the ultimate source of truth for what go.mod should look like, be it with or without Bazel. You could also try go mod tidy -e, which is more forgiving.

[jasonbdly]

That makes sense, thanks for clarifying!

The go mod tidy -e tip was super helpful as well. In fact, that command was able to accurately detect which modules are direct vs indirect dependencies based on the imports in the code, so that's awesome.