feat: Configurable OAuth2 login page

[klopfdreh]

Address the issue: spring-cloud/spring-cloud-dataflow-ui#1887

When OAuth2 is enabled and there are client-registrations with authorization-grand-type authorization_code all those are going to be listed within the Angular-Login, now. Also when you logout and login again you are redirected correctly.

You can see the appearance here: spring-cloud/spring-cloud-dataflow-ui#1923

❗ There is one issue left I couldn't solve yet. When I fill the getAuthenticatedPaths() the loginUrl can be accessed because of to many redirects - even if this.authorizationProperties.getPermitAllPaths().add(authorizationProperties.getLoginUrl()); is added. I am going to mark this as a comment.

To test OAuth2 locally I used the following configuration (important properties are login-url: "/dashboard/#/authentication-required" and the client registration user_login):

spring:
  security:
    oauth2:
      resourceserver:
        opaquetoken:
          introspection-uri: http://localhost:8080/api/introspect
          client-id: 'vsrc13725ClientId'
          client-secret: 'vsrc13725ClientId'
      client:
        registration:
          user_login:
            provider: user_login
            client-id: 'vsrc13725ClientId'
            client-secret: 'vsrc13725Secret'
            client-authentication-method: client_secret_basic
            redirect-uri: http://localhost:9393/login/oauth2/code/user_login
            authorization-grant-type: authorization_code
            scope:
              - read
              - write
              - admin
          program_login:
            provider: program_login
            client-id: 'vsrc13725ClientId'
            client-secret: 'vsrc13725Secret'
            authorization-grant-type: client_credentials
            scope:
              - read
              - write
              - admin
        provider:
          user_login:
            token-uri: http://localhost:8080/oauth/token
            user-info-uri: http://localhost:8080/api/users/me
            user-name-attribute: email
            authorization-uri: http://localhost:8080/oauth/authorize
          program_login:
            token-uri: http://localhost:8080/oauth/token
  cloud:
    dataflow:
      security:
        authorization:
          defaultProviderId: program_login
          provider-role-mappings:
            user_login:
              map-oauth-scopes: true
              parse-oauth-scope-path-parts: false
              role-mappings:
                ROLE_CREATE: 'write'
                ROLE_DEPLOY: 'write'
                ROLE_DESTROY: 'write'
                ROLE_MANAGE: 'admin'
                ROLE_MODIFY: 'write'
                ROLE_SCHEDULE: 'write'
                ROLE_VIEW: 'read'
            program_login:
              map-oauth-scopes: true
              parse-oauth-scope-path-parts: false
              role-mappings:
                ROLE_CREATE: 'write'
                ROLE_DEPLOY: 'write'
                ROLE_DESTROY: 'write'
                ROLE_MANAGE: 'admin'
                ROLE_MODIFY: 'write'
                ROLE_SCHEDULE: 'write'
                ROLE_VIEW: 'read'
          login-url: "/dashboard/#/authentication-required"
          permit-all-paths: "/management/health,/management/health/liveness,/management/health/readiness,/management/info,/security/info,/assets/**,/dashboard/logout-success-oauth.html,/favicon.ico"
          rules:
            # About

            - GET    /about                          => hasRole('ROLE_VIEW')

            # Audit

            - GET /audit-records                     => hasRole('ROLE_VIEW')
            - GET /audit-records/**                  => hasRole('ROLE_VIEW')

            # Boot Endpoints

            - GET /management/**                  => hasRole('ROLE_MANAGE')

            # Apps

            - GET    /apps                           => hasRole('ROLE_VIEW')
            - GET    /apps/**                        => hasRole('ROLE_VIEW')
            - DELETE /apps/**                        => hasRole('ROLE_DESTROY')
            - POST   /apps                           => hasRole('ROLE_CREATE')
            - POST   /apps/**                        => hasRole('ROLE_CREATE')
            - PUT    /apps/**                        => hasRole('ROLE_MODIFY')

            # Completions

            - GET /completions/**                    => hasRole('ROLE_VIEW')

            # Job Executions & Batch Job Execution Steps && Job Step Execution Progress

            - GET    /jobs/executions                => hasRole('ROLE_VIEW')
            - PUT    /jobs/executions/**             => hasRole('ROLE_MODIFY')
            - GET    /jobs/executions/**             => hasRole('ROLE_VIEW')
            - GET    /jobs/thinexecutions            => hasRole('ROLE_VIEW')

            # Batch Job Instances

            - GET    /jobs/instances                 => hasRole('ROLE_VIEW')
            - GET    /jobs/instances/*               => hasRole('ROLE_VIEW')

            # Running Applications

            - GET    /runtime/streams                => hasRole('ROLE_VIEW')
            - GET    /runtime/streams/**             => hasRole('ROLE_VIEW')
            - GET    /runtime/apps                   => hasRole('ROLE_VIEW')
            - GET    /runtime/apps/**                => hasRole('ROLE_VIEW')

            # Stream Definitions

            - GET    /streams/definitions            => hasRole('ROLE_VIEW')
            - GET    /streams/definitions/*          => hasRole('ROLE_VIEW')
            - GET    /streams/definitions/*/related  => hasRole('ROLE_VIEW')
            - GET    /streams/definitions/*/applications  => hasRole('ROLE_VIEW')
            - POST   /streams/definitions            => hasRole('ROLE_CREATE')
            - DELETE /streams/definitions/*          => hasRole('ROLE_DESTROY')
            - DELETE /streams/definitions            => hasRole('ROLE_DESTROY')

            # Stream Deployments

            - DELETE /streams/deployments/*          => hasRole('ROLE_DEPLOY')
            - DELETE /streams/deployments            => hasRole('ROLE_DEPLOY')
            - POST   /streams/deployments/**         => hasRole('ROLE_MODIFY')
            - GET    /streams/deployments/**         => hasRole('ROLE_VIEW')

            # Stream Validations

            - GET /streams/validation/               => hasRole('ROLE_VIEW')
            - GET /streams/validation/*              => hasRole('ROLE_VIEW')

            # Stream Logs
            - GET /streams/logs/**                    => hasRole('ROLE_VIEW')

            # Task Definitions

            - POST   /tasks/definitions              => hasRole('ROLE_CREATE')
            - DELETE /tasks/definitions/*            => hasRole('ROLE_DESTROY')
            - DELETE /tasks/definitions              => hasRole('ROLE_DESTROY')
            - GET    /tasks/definitions              => hasRole('ROLE_VIEW')
            - GET    /tasks/definitions/*            => hasRole('ROLE_VIEW')

            # Task Executions

            - GET    /tasks/executions               => hasRole('ROLE_VIEW')
            - GET    /tasks/executions/*             => hasRole('ROLE_VIEW')
            - POST   /tasks/executions               => hasRole('ROLE_DEPLOY')
            - POST   /tasks/executions/*             => hasRole('ROLE_DEPLOY')
            - DELETE /tasks/executions/*             => hasRole('ROLE_DESTROY')
            - DELETE /tasks/executions               => hasRole('ROLE_DESTROY')
            - GET    /tasks/info/*                   => hasRole('ROLE_VIEW')

            # Task Schedules

            - GET    /tasks/schedules                => hasRole('ROLE_VIEW')
            - GET    /tasks/schedules/*              => hasRole('ROLE_VIEW')
            - GET    /tasks/schedules/instances      => hasRole('ROLE_VIEW')
            - GET    /tasks/schedules/instances/*    => hasRole('ROLE_VIEW')
            - POST   /tasks/schedules                => hasRole('ROLE_SCHEDULE')
            - DELETE /tasks/schedules/*              => hasRole('ROLE_SCHEDULE')
            - DELETE /tasks/schedules                => hasRole('ROLE_SCHEDULE')

            # Task Platform Account List */

            - GET    /tasks/platforms                => hasRole('ROLE_VIEW')

            # Task Validations

            - GET    /tasks/validation/               => hasRole('ROLE_VIEW')
            - GET    /tasks/validation/*              => hasRole('ROLE_VIEW')

            # Task Logs
            - GET /tasks/logs/*                       => hasRole('ROLE_VIEW')

            # Task Ctr
            - GET    /tasks/ctr/*                     => hasRole('ROLE_VIEW')

            # Tools

            - POST   /tools/**                       => hasRole('ROLE_VIEW')

and in the server I switched to my locally build UI with:

pom.xml in spring-cloud-dataflow-server

		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-dataflow-ui</artifactId>
			<version>${project.version}</version>
			<exclusions>
				<exclusion>
					<artifactId>spring-cloud-dataflow-ui</artifactId>
					<groupId>org.springframework.cloud</groupId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<artifactId>spring-cloud-dataflow-ui</artifactId>
			<groupId>org.springframework.cloud</groupId>
			<version>3.3.4-SNAPSHOT</version>
		</dependency>

[klopfdreh]

Hey @onobc - I am not that good in Spring Security configuration, but I thought when I call permitAll() first and after this authenticated() for a more concrete page pattern, that those paths are going to be whitelisted. This seems to fail in my case - can you or someone of the SCDF team help me out there?

Other than that the OAuth2 Angular login is working.

[klopfdreh]

I also actually don't know why this test is failing 😨

Could it be that I changed the AboutController and the ClientRegistrations are provided in the SecurityInfo response, now?

[onobc]

Hi @klopfdreh ,
I will try to take a look sometime today.

[klopfdreh]

Thanks a lot!

[klopfdreh]

Hey @onobc - I think I found a solution - see: klopfdreh#2

The UI already checks the security in the root path. So I adjusted the settings so that the login page is allowed.

The login-url I adjusted a bit: /dashboard/index.html#/authentication-required

with this you can now login without having an impact on other services. But please verify that I did not missed anything.

Here are some screenshots what happens:

1. I entered the url http://localhost:9393/dashboard

2. The new dashboard login page with the registrations is shown 3. When I click on a registration I am redirected to the OAuth server's consent page 4. When I allowed the scopes to the backend I can see the dashboard

[klopfdreh]

One further good thing to mention is that you still can use the spring-security login page if you want to - I just forgot about this!

[onobc]

@klopfdreh thanks for the updates. I did not get a chance to get to this yesterday. I will be sure to dedicate some time early this work week to thoroughly review the proposal. Again, thank you for all you do.

[klopfdreh]

No worries - I am glad to help a bit in my free time.

[klopfdreh]

@onobc - Build is repaired 👍 - I am adjust the skipper to work the same way in SkipperOAuthSecurityConfiguration.

[klopfdreh]

The last thing I could think about is what clientRegistrations are returned from the SecurityController - currently the registrations are filtered for those which have AuthorizationGrantType set to authorization_code as those are the important to be used for the user OAuth2 login within the UI.

The standard spring-security OAuth2 login form also filters for authorization_code.

Currently I would stick to this, but this could be improved that the all clientRegistrations are returned as complex objects and the UI filters for `authorization_code.

[klopfdreh]

I tested the toggle between

login-url: "/login" (default spring-security oauth2 login)

and

login-url: "/dashboard/index.html#/authentication-required" (angular login)

both is working now.

[corneil]

@klopfdreh Thank you for this contribution.
We will be able to give it the attention it deserves once we have 2.11.0 out the door.

[klopfdreh]

All right! If there are any further questions just ping me. 👍

[klopfdreh]

Those security settings need to be checked before merging.

[klopfdreh]

Let me know when you are able to have a look at this change. Might be cool to get a fancy Angular Login page instead of simple text. 😄

[claudiahub]

Resolved the conflicts to integrate the updates for the AboutController.
Other than that the UI looks good to me.
thank you @klopfdreh

[onobc]

@klopfdreh we will need to wait post 2.11.2 for this to get in. We are busy fixing fallout from the big move to support Boot3. Once the dust settles we will get this in.

Thanks for your patience.

[klopfdreh]

@onobc - just a short reminder - would be nice to have this implemented as we can adjust our CSP settings to not allow any content from the bootstrap CDN anymore 👍 (zero trust)

Important hint: A review needs to be done before merging: #5350 (comment)

[onobc]

Hi @klopfdreh - thanks for the reminder. Not exactly sure when we will get to this but it is on the front burner again w/ your ping :)

[klopfdreh]

Awesome! 😄

[corneil]

@klopfdreh You will probably find the latest snapshot behaving a lot better.
Please let me know if you find anything weird.
Which IdP are you using?
We added an integration test for Keycloak and found that the latest version of Keycloak doesn't make scope as easily assignable to users. So we created a realm with roles and configure a mapping from those roles and assigning roles to a group and user to the group which worked as expected and is in line with how most people use Keycloak in my experience.

[klopfdreh]

We are using a custom OAuth2 integration - no Keycloak. Will the new behavior also cover the standard OAuth2 integration as well as the new with realms? That would be very important for us.

[corneil]

We are using a custom OAuth2 integration - no Keycloak. Will the new behavior also cover the standard OAuth2 integration as well as the new with realms? That would be very important for us.

This is why I asked that you test the latest snapshot. It should support all latest spring-security configuration along with the dataflow specific security configuration.

[klopfdreh]

All right - I try to create a test setup, but this will take a bit.

[klopfdreh]

I upgraded our parent to 3.0.0-SNAPSHOT to test the new OAuth2 behavior, but we can't resolve the dependency:

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-dataflow-rest-client</artifactId>
            <version>3.0.0-SNAPSHOT</version>
        </dependency>

corretly.

It resolves to:

but the classes of the packages org.springframework.cloud.dataflow.rest.client.* are missing.

I just wanted to remove AppBootSchemaVersion but more are missing.

Edit: I was able to solve it. Was a local problem. When I cleaned the cache it was working like expected:

We refactored everything so that schemaTarget is not used any longer.

[klopfdreh]

When we update to the most recent SNAPSHOT we receive the following error:

Caused by: java.lang.ClassNotFoundException: io.fabric8.kubernetes.client.KubernetesClientBuilder
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
	at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.loadClass(JarUrlClassLoader.java:107)
	at org.springframework.boot.loader.launch.LaunchedClassLoader.loadClass(LaunchedClassLoader.java:91)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
	... 28 more

We checked the version and in 5.12.4 the class is indeed missing.

This is because of spring-cloud-kubernetes-fabric8-autoconfig which chanced from version 2.1.9 to 3.2.0

I guess https://github.com/spring-cloud/spring-cloud-dataflow/blob/main/spring-cloud-dataflow-build/spring-cloud-dataflow-build-dependencies/pom.xml#L38

kubernetes-fabric8-client.version needs to be > 6.x.x and java-cfenv > 3.3.x

Edit:
When using the following dependencies all is working like expected:

        <java-cfenv.version>3.3.0</java-cfenv.version>
        <kubernetes-client-bom.version>6.13.4</kubernetes-client-bom.version>

[klopfdreh]

Spring Cloud Skipper 3.0.0-SNAPSHOT is working with Spring Boot 3.4.0, Spring Cloud Data Flow 3.0.0-SNAPSHOT I had to downgrade to spring-boot-dependencies 3.3.6 and spring-cloud-dependencies 2023.0.4 as spring-boot-autoconfigure-3.4.0.jar causing issues

***************************\nAPPLICATION FAILED TO START\n***************************
Description:
An attempt was made to call a method that does not exist. The attempt was made from the following location:
    org.springframework.cloud.dataflow.server.config.DataFlowServerConfiguration.transactionManager(DataFlowServerConfiguration.java:88)
The following method did not exist:
    'void org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers.customize(org.springframework.transaction.PlatformTransactionManager)'
The calling method's class, org.springframework.cloud.dataflow.server.config.DataFlowServerConfiguration, was loaded from the following location:
    jar:nested:/deployments/spring-cloud-data-flow-server-1.0.0-feature-scdf300test-SNAPSHOT-exec.jar/!BOOT-INF/lib/spring-cloud-dataflow-server-core-3.0.0-SNAPSHOT.jar!/org/springframework/cloud/dataflow/server/config/DataFlowServerConfiguration.class
The called method's class, org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers, is available from the following locations:
    jar:nested:/deployments/spring-cloud-data-flow-server-1.0.0-feature-scdf300test-SNAPSHOT-exec.jar/!BOOT-INF/lib/spring-boot-autoconfigure-3.4.0.jar!/org/springframework/boot/autoconfigure/transaction/TransactionManagerCustomizers.class
The called method's class hierarchy was loaded from the following locations:
    org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers: jar:nested:/deployments/spring-cloud-data-flow-server-1.0.0-feature-scdf300test-SNAPSHOT-exec.jar/!BOOT-INF/lib/spring-boot-autoconfigure-3.4.0.jar!/
\nAction:
Correct the classpath of your application so that it contains compatible versions of the classes org.springframework.cloud.dataflow.server.config.DataFlowServerConfiguration and org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers\n

[klopfdreh]

After all those changes I was able to test the OAuth2 Login and it was a bit weird.

1. The following page was shown
   
2. When I clicked on the link I was redirected to the standard page of spring-security-oauth2
   
3. Then I was logged in
   

[klopfdreh]

The login error I previously mentioned was due to an infra issue on our side - so basically the login is working, but with this weird behavior.

This weird in-between page is standard Spring Security behaviour.
That page is where multiple IdPs will show up if they exist.
When there is only one we can go straight to the single IdP.

[klopfdreh]

Hey @corneil - I saw you edited my post and responded to me.

Does the in-between page also show up if you mix client credential flow and authorization code flow as that is a normal usecase for most of the SCDF customers I think.

Like:

      security:
        oauth2:
          resourceserver:
            opaquetoken:
              introspection-uri: '...'
              client-id: '...'
              client-secret: '...'
          client:
            registration:
              user_login:
                provider: user_login
                client-id: '...'
                client-secret: '...'
                client-authentication-method: client_secret_post
                redirect-uri: '...'
                authorization-grant-type: authorization_code
                scope:
                  - '...'
              program_login:
                provider: program_login
                client-id: '...'
                client-secret: '...'
                authorization-grant-type: client_credentials
                scope:
                  - ...
            provider:
              user_login:
                token-uri: '...'
                user-info-uri: '...'
                user-name-attribute: email
                authorization-uri: '...'
              program_login:
                token-uri: '...'

Then the in-between page would be shown all the time as the IdP for client credential flow and authorization code flow is always the same.

This is required as we login with users to the ui, but also start applications with a technical user via REST-API.

[klopfdreh]

You might find this out by checking the token-uri of the provider and if this is the same for both registration - then it can be considered as one IdP