Could I add a spring boot version of endpoints-v2-backend?

[aubreyyan]

In which file did you encounter the issue?

https://github.com/GoogleCloudPlatform/java-docs-samples/tree/master/appengine-java8/endpoints-v2-backend

Did you change the file? If so, how?

I can add a new endpoints-v2-backend-spring copy that closely resembles the linked project but uses JAR instead of WAR (which imo is the better file format) in conjunction with Spring Boot.

Describe the issue

The existing demo project is fine, although it's really not beginner friendly and doesn't teach best development practices since, AFAIK, there is no way to test it locally.

Using Spring allows you to simulate what's being deployed by app engine on localhost.

I have my own more personalized version of the project working with GAE and GCF. (and of course maven + spring + jar)

[aubreyyan]

@lesv does tagging as feature request mean I can submit a pull request or does it have to go through some other story vetting first?

[lesv]

@Aubrey-Y Sorry- I didn't realize you were offering to provide this.
@averikitsch Do you want this?

[lesv]

Hi @Aubrey-Y - we talked about this today and thought you might wish to look at our community (code) pages. Your sample is likely to get greater visibility that way.

Feel free to ping me or Averi when you are ready for review. Either on the repo or on this issue.

I'm going to close this for now, feel free to reopen.

[aubreyyan]

@lesv The tutorial that specifically references this is under the codelabs subdomain (https://codelabs.developers.google.com/codelabs/cloud-endpoints-appengine/index.html?index=..%2F..index#0) so is there a way to create a tutorial there that would accompany a PR to java-docs-samples or are you saying that you would want any PR to be in the community repository specifically, and thus with just the markdown writeup?

My personal thinking is that there is a much clearer purpose for it here as opposed to community/tutorials because it's an idea spawned out of the codelabs example here, so I'm not sure making a tutorial somewhere else fixes it. I trust your best judgement though.

What I was trying to accomplish here, initially, was that if someone had run into the same issue where they would have liked to run a cloud endpoints sample locally, they could poke around and quickly find this alternate version of the project/tutorial. I would of course be happy to write a tutorial for it, even though the changes wouldn't be that significant.

[lesv]

@navinger @tomboc @frankyn @simonz130 All of you have changed this.
Do any of you own this?
I'm happy to take this PR, but I want to make sure it's actually useful and helpful.

[frankyn]

I no longer own endpoints.

[lesv]

@Aubrey-Y Are you still interested in doing this? If so, I'll chase down an owner, though it probably doesn't make sense to update an AppEngine Java 8 sample.

If you wanted to contribute a CloudRun sample that uses OpenAPI, that would be welcome.

[aubreyyan]

absolutely! where would you want it added in the repository? @lesv

[averikitsch]

Thank you for your patiences. Can you add this sample under the run/ directory and add @lesv and I as reviewers? Thanks

[aubreyyan]

sure, i dont have it prepared already but ill try to find time to do it over the next week

[aubreyyan]

One point of clarification - should it use Java 8 or another version (i.e. 11)? @lesv

(or both)

[averikitsch]

One point of clarification - should it use Java 8 or another version (i.e. 11)? @lesv

(or both)

Java 11 is preferred.

[aubreyyan]

A small update. So I started an mvp today and ran into an issue with what I assume are the discrepancies between javax.servlet (groupid servlet-api) requirements for containerized and non-containerized maven applications running Tomcat.

From what I can tell, to run tomcat with docker, servlet-api-3.1.jar+ is needed (https://stackoverflow.com/questions/60909834/an-attempt-was-made-to-call-a-method-that-does-not-exist-sts) which is a problem because Cloud Endpoints dependencies endpoints-framework and endpoints-management-control-appengine-all both come prepackaged with servlet-api-2.5.jar which I wasn't able to exclude in my pom (I tried, but servlet-api-2.5.jar kept coming back with endpoints-management-control-appengine-all).

cloudendpoints, which I assume is under the @google org, manages these dependencies at cloudendpoints/endpoints-management-java and I've opened up an issue there to upgrade their servletApiVersion cloudendpoints/endpoints-management-java#57 but I have my doubts about getting a response, given the last commit to master was 7 months ago and their last release was in 2017.

In the meantime, I'll keep looking for ways to ignore their version of servlet-api, but in my mind the best outcome would be updating the versioning on their end. If you guys are able to backchannel directly with cloudendpoints org, that would be nice as well!

If you have any other ideas I'd be open to those.

[aubreyyan]

In the meantime, I actually figured out a solution.

Even though these are the dependencies used in the legacy appengine java8 endpoints sample that I'm sort of trying to replicate,

java-docs-samples/appengine-java8/endpoints-v2-backend/pom.xml

Lines 47 to 59 in 2eca87d

	<!-- Compile/runtime dependencies -->
	<dependency>
	<groupId>com.google.endpoints</groupId>
	<artifactId>endpoints-framework</artifactId>
	<version>${endpoints.framework.version}</version>
	</dependency>
	<!-- [START api_management] -->
	<dependency>
	<groupId>com.google.endpoints</groupId>
	<artifactId>endpoints-management-control-appengine-all</artifactId>
	<version>1.0.12</version>
	</dependency>
	<!-- [END api_management] -->

I had a peek through Google's other endpoints-related libraries in mvnrepository and found a version of endpoints management similar to endpoints-management-control-appengine-all because after all, we're not using app engine anymore.

The library endpoints-management-control actually has a package structure that enables the <exclusions> option to work, similar to endpoints-framework, so I did a dry run-through with it and it works the same way.

So, I can move forward for now like this (even though it looks ugly)

        <!-- Compile/runtime dependencies -->
        <dependency>
            <groupId>com.google.endpoints</groupId>
            <artifactId>endpoints-framework</artifactId>
            <version>${endpoints.framework.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- [START api_management] -->
        <dependency>
            <groupId>com.google.endpoints</groupId>
            <artifactId>endpoints-management-control</artifactId>
            <version>1.0.12</version>
            <exclusions>
                <exclusion>
                    <groupId>javax.servlet</groupId>
                    <artifactId>servlet-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- [END api_management] -->

[aubreyyan]

How strictly do we want to follow the pipeline orchestrated by appengine-java8/endpoints-v2-backend? @lesv

java-docs-samples/appengine-java8/endpoints-v2-backend/pom.xml

Lines 108 to 118 in 2eca87d

	<!-- [START endpoints_plugin] -->
	<plugin>
	<groupId>com.google.cloud.tools</groupId>
	<artifactId>endpoints-framework-maven-plugin</artifactId>
	<version>2.0.1</version>
	<configuration>
	<!-- plugin configuration -->
	<hostname>${endpoints.project.id}.appspot.com</hostname>
	</configuration>
	</plugin>
	<!-- [END endpoints_plugin] -->

I did some digging into the functionality provided by the Endpoints Framework Maven plugin (documentation) and it doesn't look like it supports much else other than generating a basic openapi spec of your controllers in json format.

This is a limitation in terms of a demo project, because it doesn't really allow for much showing off of some of the most interesting features of Cloud Endpoints, like api keys for example.

Google does have documentation regarding how to enable api keys but it won't fit nicely into the plugin generator because of two reasons:

1. the documentation for adding api keys is in yaml format, which imo most people would prefer to json, but the plugin generates in json still
2. there is no option, command line or otherwise, for the plugin to turn on api keys (which is actually just a few more fields) so the user would have to edit it in manually - after every mvn endpoints-framework:openApiDocs command

I just feel like it's not a good look to the end user if we try showing off literally anything else other than the "track api activity" page, because of these constraints i just mentioned, which is all the existing demo is doing.

I know, I know, this was a sort of long read, but we have a few options:

1. possibly explore open source alternatives outside of google? I know there are a bunch of other libraries that can generate openapi documents, but I don't know what the rules are about going outside of the google ecosystem
2. put in some feature requests for the endpoints framework plugin to be updated to support maybe enabling api keys? this would be a bit difficult because I tracked down the library that actually evaluates the command created by the plugin and it seems like it's closed source, so someone at google would have to implement it. I've put in an issue over at cloudendpoints Is appengine-tools-sdk (used in this repo) closed source? cloudendpoints/endpoints-java#190 inquiring about it but since it's not a high-volume org/repo in general I don't think much can be done on this front
3. just go with status quo and use the limited features of the endpoints framework plugin to show off the api activity page

[averikitsch]

The google endpoint framework is for App Engine which uses servlets. We wouldn't want to force integrating it with Spring Boot. We want to show how to use Endpoints backed by Extensible Service Proxy V2 Beta (ESPv2 Beta) since it's the preferred way. Therefore the best use of time would be to provide a Spring Boot app with the proper tools to generate the openapi.yaml see in Configuring Endpoints. We then could point that documentation to your code.

[aubreyyan]

okay, im just now realizing this - so no more endpoints related dependencies in my pom now.

provide a Spring Boot app with the proper tools to generate the openapi.yaml

To make sure I'm understanding you correctly, you're looking for the spring boot app to come with a tool to generate an openapi-run.yaml file? There wasn't anything in the documentation you linked that references any kind of library that would have that kind of functionality, to generate openapi-run.yaml. I could try my hand at writing one, of course, but that's probably outside the scope of a simple example app?

From reading the documentation though the implication was that you'd have to write and maintain it manually.

[averikitsch]

I though you had said you had your personal project working with Spring and Endpoints on App Engine, correct? Moving the sample that uses the Endpoint framework with servlets to the Spring framework shouldn't work because the Endpoints framework only works on our Java 8 App Engine (servlet framework) runtime.

To use Java 11 on App Engine or Cloud Run, all you need is the openAPI spec and both these runtimes work with Spring. You could provide a sample + the OpenAPI config file like the docs say; however it would be more interesting to have the spec generated. There should be libraries to generate OpenAPI config files from Spring apps. Nothing about this is actually Google or endpoints related, it's just based on the OpenAPI spec which is not Google owned.

[aubreyyan]

So third party libraries/plugins are fine?

[averikitsch]

So third party libraries/plugins are fine?

Yes, if you have any hesitations or questions about vetting the library/plugins just let us know here.

[aubreyyan]

Follow up -> the Configuring Endpoints resource you linked (https://cloud.google.com/endpoints/docs/openapi/get-started-cloud-run#endpoints_configure) uses Swagger 2.0. Can we upgrade to Swagger 3.0.1, and I guess a relevant question is also whether it's supported by Cloud Endpoints? @averikitsch

[averikitsch]

Unfortunately we don't yet support version 3.

[aubreyyan]

If it helps at all, I recorded a briefish 15m video walking through the PR, while I was half asleep, with one hand ><

So it's definitely low quality but if you need some context it's there
https://www.loom.com/share/761b2d48a74d46538f257dd6fb01638b

At a high level, I found an open source library (springdoc) that supports openapi3, which is actually not all that different from the configuration google needs. With that said, I've tried and gcp won't accept openapi3.

So there are a few minute differences, mostly lines that need to be removed, in order to convert an openapi3 file to swagger2.

Since springdoc already had a convenient plugin that generates an openapi.yaml file during the integration test phase, I wrote another plugin that can conveniently run right after springdoc generates the openapi3 version, and seamlessly deliver a swagger2 version that gcp accepts (I dont actually remember to demonstrate it in the video but basically mvn verify or mvn clean install will create a valid openapi yaml file in the project root directory).

I've uploaded it here for now: https://github.com/aubrey-y/openapi-converter-maven-plugin with GCP package structure and license javadocs, under the com.google.cloud.tools groupId, in anticipation that if you used a plugin like this, you'd want to own it, but I know that comes with its own set of issues because then GCP would sort of still be in charge of maintaining it. To test it locally with my fork all you need to do is clone the plugin and run mvn clean install, which loads it into your local m2 repository.

I don't mind if I transfer ownership to you guys and you publish the plugin, or if I try to publish it myself.

Or, maybe this custom plugin thing is not a good idea? It works seamlessly with springdoc to generate the openapi file as was initially requested though, so I like it.

One thing that I am stuck on though is the ESPv2 container.

I try and follow through all the steps from scratch here https://www.loom.com/share/e3e33e8ad1434a4888243360b92a7f6f and the ESP container never worked, so I don't think I'll be able to figure that out without extra help. I can document all the local deployment steps but I don't think I'm understanding the tutorial correctly.

[aubreyyan]

A nice thing about this approach could also be that if Cloud Endpoints eventually supports openapi3, all you would have to do is remove the extra conversion plugin and you would already be at openapi 3, less a few configurations possibly.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.