Document content_roots for prefix removal

[kriswuollett]

Perhaps it is just a configuration mistake, but I suspect protofetch works on the assumption that protos are stored at the root of a git repository. I'm using proto_out_dir = "protos" with the following dependency:

[grpc_health_v1]
url = "github.com/grpc/grpc"
revision = "b8a04acbbf18fd1c805e5d53d62ed9fa4721a4d1" # v1.64.0
protocol = "https"
allow_policies = ["src/proto/grpc/health/v1/*"]

Instead of finding a file at protos/grpc/health/v1/health.proto, it is actually at protos/src/proto/grpc/health/v1/health.proto. I assumed maybe protofetch had the logic to automatically strip prefixes based on the proto package name.

I tried using url = "github.com/grpc/grpc/src/proto" as a configuration option to see if something like that was undocumented, but instead I get this error:

ERROR Could not build a valid descriptor from a protofetch toml file due to err Missing url component `forge` in string `github.com/grpc/grpc/src/proto`
ERROR IO error reading configuration toml: No such file or directory (os error 2)

If one wants to compile multiple unrelated proto dependency trees at once, I believe they need to be rooted in the same directory? Is there a way I am currently supposed to use protofetch so all my dependencies could end up in the same logical & physical directory tree?

For example adding this to my config:

[envoy_api_v2]
url = "github.com/envoyproxy/envoy"
revision = "72d653e2540cc5f77e2acdc1c9a57a10263d74dc"
protocol = "https"
allow_policies = ["api/envoy/api/v2/*"]

Fetching it (155 MB cached, 53 seconds):

% time protofetch fetch
INFO Resolving github.com/grpc/grpc
INFO Resolving github.com/envoyproxy/envoy
INFO Wrote lockfile to /Users/kris/Code/appbiotic/corp/protofetch.lock
INFO Fetching dependencies source files...
INFO Copying proto files from corp descriptor...
INFO Creating new worktree for envoy_api_v2 at /Users/kris/.cache/protofetch/dependencies/envoy_api_v2/72d653e2540cc5f77e2acdc1c9a57a10263d74dc.
INFO Found existing worktree for grpc_health_v1 at /Users/kris/.cache/protofetch/dependencies/grpc_health_v1/b8a04acbbf18fd1c805e5d53d62ed9fa4721a4d1.
protofetch fetch  15.70s user 3.55s system 36% cpu 53.142 total

The QuicProtocolOptions message is inprotos/api/envoy/api/v2/listener/quic_config.proto not protos/envoy/api/v2/listener/quic_config.proto.

[rtimush]

If I'm not mistaken, that's what content_roots are for. Unfortunately, they seem to be undocumented.

[kriswuollett]

If I'm not mistaken, that's what content_roots are for. Unfortunately, they seem to be undocumented.

Thanks. Seemed like it should work, but in the following example the grpc health protos are missing from the tree, but otherwise it looks like it stripped the prefix of just envoy protos:

name = "corp"
proto_out_dir = "protos"

[grpc_health_v1]
url = "github.com/grpc/grpc"
revision = "b8a04acbbf18fd1c805e5d53d62ed9fa4721a4d1" # v1.64.0
protocol = "https"
allow_policies = ["src/proto/grpc/health/v1/*"]
content_roots = ["src/proto"]

[envoy_api_v2]
url = "github.com/envoyproxy/envoy"
revision = "72d653e2540cc5f77e2acdc1c9a57a10263d74dc"
protocol = "https"
allow_policies = ["src/api/envoy/api/v2/*"]
content_roots = ["src/api"]

It's difficult to try different variations due to #27 without creating minimal sample repos -- I can't just delete the source tree.

[kriswuollett]

Perhaps my use case is not supported as I don't have "any protos of my own"?

protofetch/README.md

Lines 140 to 142 in bf7ebd8

	This is helpful especially when you take advantage of the pruning feature which allows you to only recursively fetch
	the proto files you actually need. With pruning enabled, protofetch will recursively find what protofiles your root
	protos depend on and fetch them for as long as they are imported (flag as transitive dependency or fetched from other modules).

Use case is building a server that provides two services, the grpc health protocol for status checks, and a sidecar for Envoy.

I could specify my dependency roots from files, but realistically I'd just need to list fully qualified gRPC service names.

[kriswuollett]

This works good enough for now even though it fetches more than I need, but I think this prefix on its own is just a documentation issue as mentioned above in #138 (comment):

name = "corp"

[envoy]
url = "github.com/envoyproxy/envoy"
revision = "72d653e2540cc5f77e2acdc1c9a57a10263d74dc"
protocol = "https"
allow_policies = ["envoy/*"]
content_roots = ["api"]

[grpc_health_v1]
url = "github.com/grpc/grpc"
# v1.64.0
revision = "b8a04acbbf18fd1c805e5d53d62ed9fa4721a4d1"
protocol = "https"
allow_policies = ["grpc/health/v1/*"]
content_roots = ["src/proto"]