Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to configure custom module paths #53

Open
pixlcrashr opened this issue Apr 2, 2021 · 11 comments
Open

Option to configure custom module paths #53

pixlcrashr opened this issue Apr 2, 2021 · 11 comments

Comments

@pixlcrashr
Copy link

pixlcrashr commented Apr 2, 2021

Hi there!

Is it currently possible to provide a custom module path to a protobuf package?

For example, when generating e.g. Go or C# files, it is possible to specify a custom module location/namespace using options:

syntax = "proto3";

package my.package;

option go_package = "github.com/my/package";
option csharp_namespace = "Protobuf.My.Package";

// and this is what I imagine for ngx-grpc:
// option ngx_module = "@my/package";
...

If this is already implemented, please tell what the regarding option is called and if it is not implemented, would you like to do so?

Best regards,
Vincent

@smnbbrv
Copy link
Owner

smnbbrv commented Apr 6, 2021

hi,

so far this is not supported.

Could you please describe your idea? Is this for sharing it as a library or what is this for?

@pixlcrashr
Copy link
Author

pixlcrashr commented Apr 6, 2021

@smnbbrv

hi,

so far this is not supported.

Could you please describe your idea? Is this for sharing it as a library or what is this for?

General idea

Sure thing:

My idea is about, in fact, having the ability to create Angular packages when using multiple protobuf package definitions.
Currently, if a protobuf package requires a package from a parent directory, the according TypeScript imports are relative to the TS file, respectively.
For example, I have a protobuf package file called "A.proto" and a protobuf package file "subdir/B.proto", which imports messages from "A.proto", then the generated TypeScript file for "B.proto" will currently use relative import paths like this:

import { ... } from '../a.pb';

Problem when creating an Angular package

Now, since a single Angular package is not able to provide TypeScript submodules due to its architecture, the only way is to separate each subdir containing protobuf generated files into an Angular sub package using ng-packagr.

Thus, when splitting the generated files into multiple Angular sub-packages, we want to reuse code from other Angular packages that were created before. Code reuse is not possible when using relative paths, as files would be re-included multiple times thus increasing the package size and making things a bit more unmaintainable (multiple types for the same data structure).

Also note that it is a technical limitation of an Angular package to include files from a parent directory, relative to the respective package.json file.
That in fact means, when having a file tree like below, that B.pb.ts CAN NOT include the path ../A.pb as that file would be outside of the sub-package. Instead, that module has to be included using @ng-mypackage as a dependency. Since circular dependencies are not possible for protocol buffers, this problem is easily solvable.

To stick to the example from above:

@ng-mypackage/
    A.pb.ts
    A.pbsc.ts
    package.json
    subdir/
        B.pb.ts
        B.pbsc.ts
        package.json

Now files could get imported from the following path

import * as a from '@ng-mypackage'
import * as b from '@ng-mypackage/subdir';

Solution

As mentioned above, I would like to have a new option field for protocol packages like it is already provided by other proto generators such as the Go (option go_package = "github.com/my/package";)
or .Net (option csharp_namespace = "Protobuf.My.Package";) generators.

I suggest to name this option option ngx_module = "@my/package";.

Note on protobuf package option fields: Protobuf options are specified to be optional by default. Introducing that module option would not break any files generated by ngx-grpc.

I hope I elaborated the idea, problem & solution understandable.

Thanks in advance.

@smnbbrv
Copy link
Owner

smnbbrv commented Apr 6, 2021

Thank you very much for detailed clarification!

Yes, I see the problem. I think I need some time to think it through...

Just a note: in case we'd need some custom option, it would be a good idea to submit a request to add it to the global list.

@pixlcrashr
Copy link
Author

Just a note: in case we'd need some custom option, it would be a good idea to submit a request to add it to the global list.

I didn't knew about that list!

@smnbbrv
Copy link
Owner

smnbbrv commented Apr 6, 2021

:) yep, protobuf is really deep...

actually, I thought many times to have custom options, but this step seems a bit heavy to perform. That's why normally all the customizations that are possible in ngx-grpc are done in the config file. However, this case cannot be solved with config file => probably it's a good idea to think of registering the option.

@smnbbrv
Copy link
Owner

smnbbrv commented Apr 6, 2021

Still, it waits. It's not yet clear how to fully handle the npm package structure + how to refer within this package + how to deal with repeating names in exports etc. Let's give it some time

@pixlcrashr
Copy link
Author

But tbh the option I imagine wouldn't be an extension. As far as I know, package options are only available within the compile time.

@pixlcrashr
Copy link
Author

Still, it waits. It's not yet clear how to fully handle the npm package structure + how to refer within this package + how to deal with repeating names in exports etc. Let's give it some time

I'll create an example with deps and a simple package structure later on today.

@pixlcrashr
Copy link
Author

pixlcrashr commented Apr 6, 2021

@smnbbrv Compiling a package using ng-packagr works fine without any problem. See my example repo: https://github.com/TheMysteriousVincent/ngx-grpc-library-example

Since ng-packagr supports intra-dependencies, creating such a library is not a problem.

Of course, there should be a script to automize creating the public-api.ts and ng-package.json files in the same order as the .proto files are ordered in.


Your uncertainities

It's not yet clear how to fully handle the npm package structure + how to refer within this package + how to deal with repeating names in exports etc.

Let's split this up:

  1. It's not yet clear how to fully handle the npm package structure

    It surely is. See the example repository. When creating protocol buffers, the user should use a unique protobuf package name for a single folder.

  2. how to refer within this package

    Within a package, let's say @ngx-grpc-library-example/test/b_package, intra-dependencies such as "A" are referred to as @ngx-grpc-library-example/test and "C" would be referred to as @ngx-grpc-library-example/test/b_package/c_package. Of course those values would depend on the value of the ngx_module option.

  3. how to deal with repeating names in exports etc. Let's give it some time

    According to the protobuf spec, repeating names of data structures within a protobuf package are not possible, nor intended.

    Name collisions can only arise, when not the same protobuf package name is used within a proto folder OR when a datatype has the same name as a service within the same protobuf package, after it was generated:

message AMessageServiceClient {}

service AMessageService {}

Both name collision sources exist because of bad protobuf design. When those name collisions arise, the user should rethink his protobuf names.

So, it is up to the user to create a false/duplicate entry. Since creating Angular libraries for gRPC services is a more advanced topic, we can definetely assume that the user knows what he does.

@pixlcrashr pixlcrashr changed the title [Question] Option to configure custom module paths Option to configure custom module paths Apr 6, 2021
@pixlcrashr
Copy link
Author

@smnbbrv Any news on this?

@pixlcrashr
Copy link
Author

@smnbbrv Any news on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants