Replace laravel auth app with mock-oauth2-server running in docker #2813

patcon · 2022-05-18T22:51:59Z

Closes: #2755

Supercedes: #2716 and #2741

This PR removes our old auth app and replaces it with a mock oauth2 server. Best way to review is to step through commits, as the removal commit is quite a bit noisier than the rest :)

Screencast

Link to full video

Steps to review

checkout branch
copy new lines from api/.env.example into api/.env (override prior local auth values)
run docker-compose up --detach --build to spin up the new mock-auth container (and rebuild php container)
~~add this line to your hosts file: 127.0.0.1 auth.talent.canada.test~~
run docker-compose run --rm maintenance bash refresh_api.sh to install new composer dependencies

To Dos

petertgiles · 2022-05-19T13:49:41Z

~~Nice, looking forward to trying this out! It uses bcmath? Any confirmation on whether we can put this on our production server?~~
Oh wait, I guess the mock auth won't have to run on a production server. 😆

esizer · 2022-05-19T14:19:03Z

Just dropping in to say this is cool! 😎 Reviewed to try and learn a bit more about how we are handling auth 👍

patcon · 2022-05-19T16:59:26Z

@tristan-orourke requested review from tristan, since there's UAT server steps, but anyone else is welcome to try before we merge. cc @petertgiles

patcon · 2022-05-19T17:01:59Z

Oh wait, I guess the mock auth won't have to run on a production server.

@petertgiles no no, you're right: bcmatch (or gmp) php extensions are something the apiapp code now needs to run some cryptographic operations quickly. This allows us to avoid requiring the arcane x5c prop (aka using a full chain cert). Even google's own jwks endpoint doesn't offer a full cert in x5c, and instead builds the public key from n (modulus) and e (exponent): https://www.googleapis.com/oauth2/v3/certs

We now assemble the public key from the modulus and exponent, instead of needing a PEM format cert in x5c (which was an uncommon approach).

infrastructure/conf/mock-auth-server.json

petertgiles

Whoah, I love this so much. ❤️ ❤️ ❤️. Another patcon wizard PR. I love seeing +321 −31,458 at the top of the PR page. I've got a few "questions, concerns, threats" below but I'm so excited for this.

api/app/Services/OpenIdBearerTokenService.php

api/composer.json

docker-compose.yml

api/app/Services/OpenIdBearerTokenService.php

infrastructure/conf/mock-auth-server.json

maintenance/README.md

infrastructure/php-container/Dockerfile

patcon · 2022-05-20T07:42:17Z

This is a killer review peter! Will get on this tomorrow :)

api/composer.json

patcon · 2022-05-21T10:23:09Z

K, I think I address all the feedback, or am waiting for a response on that one part 🙌

patcon · 2022-05-25T21:07:23Z

@petertgiles Ok, figured out how to get mock-oauth2-server working behind apache proxy. It's a little more than I expected, and I need to work outside htaccess to get proxy working, but lemme know if it looks ok!

f3634b7...b50b07d

The main confusing bit (for which I tried to document in case anyone wants to unwind what's happening later), is that both the API code (running inside the container) and the host browser (running outside the container) need to be able to find the auth server at the same url. It's explained best in comments within infrastructure/php-container/mock-auth-proxy.apache-site.conf, so i won't repeat myself here :)

patcon · 2022-05-25T22:41:35Z

My attempt to document our auth flow a little bit (WIP):

sequenceDiagram
    autonumber
    participant browser as Browser (User)
    participant admin as React Admin App
    participant api as Laravel API App
    participant auth as Auth Server (Mock)

    Note right of auth: localStorage:{stored_locale: "fr"}
    Note right of auth: cookies: none
    browser->>admin: GET /admin
    admin-->>browser: 200 OK. App HTML & JS.

    Note left of browser: check localStorage.stored_locale
    browser->>browser: /fr/admin
    Note left of browser: check localStorage.id_token (null)
    Note left of browser: render Login button

    browser->>api: GET /login?from=/fr/admin&locale=fr
    api-->>browser: 302 Found. Set-Cookie: api_session=<...>, Location: <...>
    Note right of auth: cookies: api_session=<...>
    Note left of browser: Redirect

    browser->>auth: GET /oxauth/authorize? ... {client_id,redirect_uri,response_type,scope,state,nonce,acr_values,ui_locales}
    Note right of auth: checks client_id (no-op on mock)
    auth-->>browser: 200 OK. Login Form.

    Note left of browser: Form field: [email protected]
    browser->>auth: POST /oxauth/authorize? ... {client_id,redirect_uri,response_type,scope,state,nonce,acr_values,ui_locales} [email protected]
    Note right of auth: validates user (no-op on mock)
    auth-->>browser: 302 Redirect. Location: <...>
    Note left of browser: Redirect

    browser->>api: GET /auth-callback?code=...&state=...
    Note right of api: check cookie: api_session
    api-->>browser: 302 Redirect. Location: <...>
    Note left of browser: Redirect

    browser->>admin: GET /fr/admin? ... {token_type,id_token, access_token, refresh_token,expires_in}
    admin-->>browser: 200 OK.
    Note right of auth: localStorage:{stored_locale: "fr", id_token: "<...>"}

    browser->>browser: /fr/admin/dashboard
    Note left of browser: check localStorage.id_token (present)
    Note left of browser: render Logout button

    browser->>api: POST /graphql operationName=getMe
    Note right of api: TODO token validation
    api-->>browser: 200 OK.
    Note left of browser: render admin menu.

    browser->>api: POST /graphql operationName=me
    api-->>browser: 200 OK.
    Note left of browser: render dashboard.


    browser->>api: POST /graphql operationName=getPools
    api-->>browser: 200 OK.
    Note left of browser: render pool list.

Open Questions

Is the API server using the cookie api_session or XSRF-TOKEN to track the returning user one their way back from auth server?

patcon · 2022-05-25T23:35:56Z

aside from my stretch goal of writing some high level docs in response to some of peter's questions, this is ready for review! (i also stubbed out some ideas in #2671 for end-to-end tests the would exercise some security edge-cases and ensure we handle those

petertgiles · 2022-05-26T14:27:07Z

I'm having some trouble with this. I'm getting a 503 on the authorize request. Any idea what the trouble is? I rebuilt my images and containers and reran setup.sh.

Here are my Apache logs:

[Thu May 26 14:19:30.136440 2022] [proxy:error] [pid 19] (111)Connection refused: AH00957: http: attempt to connect to 172.19.0.2:8000 (mock-auth) failed
[Thu May 26 14:19:30.136464 2022] [proxy_http:error] [pid 19] [client 172.19.0.1:48996] AH01114: HTTP: failed to make connection to backend: mock-auth, referer: http://localhost:8000/en/admin
172.19.0.1 - - [26/May/2022:14:19:30 +0000] "GET /oxauth/authorize?client_id=fake-development-client-key&redirect_uri=http%3A%2F%2Flocalhost%3A8000%2Fauth-callback&response_type=code&scope=openid+offline_access&state=ikKZ0Gp4LesN2YJNcs8KejEEds56qbcDW5vpqP2W&nonce=PfMto2iKNoZlybwzFo1J0IJPC8uF1xp0GMF8xHhs&acr_values=gckey&ui_locales=en-CA+en HTTP/1.1" 503 566 "http://localhost:8000/en/admin" "Mozilla/5.0 (X11; Linux x86_64; rv:100.0) Gecko/20100101 Firefox/100.0"

I think it's a port mixup?

tristan-orourke · 2022-05-26T14:55:48Z

Is the API server using the cookie api_session or XSRF-TOKEN to track the returning user one their way back from auth server?

The api_session is used only to choose where to redirect. If the api_session cookie disappeared between going to the auth server and coming back, authentication should still work fine.

I don't think we're using XSRF-TOKEN at all? Laravel may be setting it by default.

tristan-orourke · 2022-05-26T14:59:36Z

By the time I've found time to read through this, I think Peter has already asked all the really important questions!

I'm really curious to understand the proxy better, I'll find some time to talk through a few questions about it.

patcon · 2022-05-26T20:38:02Z

Thanks @petertgiles! I'm spinning up fresh and investigating.

Musings: wondering if it has to do with us using links in docker-compose.yml rather than networks (which i think supercedes links). I had been toying with that during my trying to get it working, and maybe something persisted that made it work for me but not you 🤔

Also, since links is deprecated, maybe our docker-compose version matters (or even if we're using docker compose (the docker-integrated command) vs docker-compose (standalone).

What version and commands are you using? I'm using docker-compose, and these versions:

patcon at Patricks-MacBook-Pro in ~/repos/gc-digital-talent (feature/2722-make-email-editable-and-nullable)
$ docker-compose --version
docker-compose version 1.29.2, build 5becea4c

patcon at Patricks-MacBook-Pro in ~/repos/gc-digital-talent (feature/2722-make-email-editable-and-nullable)
$ docker --version
Docker version 20.10.13, build a224086

patcon at Patricks-MacBook-Pro in ~/repos/gc-digital-talent (feature/2722-make-email-editable-and-nullable)
$ docker compose version
Docker Compose version v2.3.3

patcon · 2022-05-26T20:48:11Z

infrastructure/php-container/mock-auth-proxy.apache-site.conf

+#     REQUEST FROM HOST BROWSER:
+#     localhost:8000 (host) => localhost:80   (php container) => mock-auth:8080 (mock-auth container)
+#
+#     REQUEST FROM API APP:
+#                              localhost:8000 (php container) => mock-auth:8080 (mock-auth container)


@tristan-orourke this is the comment where I explain the logic. Hit me up if this and the surrounding context needs any additional discussion!

patcon · 2022-05-26T20:52:31Z

https://recordit.co/pCmAXSuZp2

Hm. Not sure what's up @petertgiles, I built fresh into /tmp, and just ran this, and it works as above:

cd /tmp
git clone GCTC-NTGC/gc-digital-talent
cd gc-digital-talent
git pr checkout 2813
docker-compose up --detach --build
docker-compose run --rm maintenance bash setup.sh

tristan-orourke · 2022-05-26T21:33:03Z

Just added "run docker-compose run --rm maintenance bash refresh_api.sh to install new composer dependencies" to Steps to Test, because I originally forgot to do that.

tristan-orourke · 2022-05-26T21:36:29Z

Hmm, it works fine fine for me, no problems

petertgiles

Hmm, I'm not sure what went wrong for me. I tried again today (maybe more carefully?) and it ran fine! I also deployed it to the dev server and it seems OK. It's still up there now if you want to poke at it.

petertgiles · 2022-05-27T13:04:03Z

Here are my tips if you want to include them in your "help people to get their environment going again" post:

delete the auth folder manually before checking out
do docker-compose down before checking out
add --build to your docker-compose up -d

patcon · 2022-05-27T17:19:53Z

k, thanks peter! Adapted from your recommendations (but favouring after-pull steps in case ppl don't notice msg)

(added auth/ to gitignore, so auth won't risk being recommitted, and unmerged feature branches still work)

After pull:

docker-compose down --remove-orphans
docker-compose up --detach --rebuild

EDIT: copy the new auth settings from api/.env.example into .env:

 # for mock oauth testing
 AUTH_SERVER_ROOT="http://localhost:8000/oxauth"
 OAUTH_URI="http://localhost:8000/oxauth/authorize"
 OAUTH_TOKEN_URI="http://localhost:8000/oxauth/token"
 OAUTH_API_CLIENT_ID="fake-development-client-key"
 OAUTH_API_CLIENT_SECRET="ignored-secret"

patcon added 3 commits May 18, 2022 18:41

Removed requirement for x5c key in jwks.

4c109df

Spike for getting mock-oauth2-server working.

de4304e

Removed all traces of laravel auth app.

5a70909

patcon changed the title ~~Mock oauth2 server alt~~ Replace laravel auth app with mock-oauth2-server running in docker May 18, 2022

patcon mentioned this pull request May 18, 2022

Get rid of custom laravel auth app #2755

Closed

6 tasks

patcon added 2 commits May 18, 2022 19:01

Oops. Enabled bcmath php extension in CI.

23850a1

Added n and e jwks key props for tests, and removed x5c.

24e3bdf

patcon self-assigned this May 19, 2022

patcon requested a review from tristan-orourke May 19, 2022 16:58

patcon commented May 19, 2022

View reviewed changes

infrastructure/conf/mock-auth-server.json Outdated Show resolved Hide resolved

Make token claims returned from mock auth more clearly fake uuids.

79e58db

petertgiles requested changes May 19, 2022

View reviewed changes

patcon mentioned this pull request May 20, 2022

Add e2e tests and commands for auth flows on devel #2719

Merged

12 tasks

yonikid15 mentioned this pull request May 20, 2022

Remove reliance on x5c property at /jwks #2716

Closed

This was referenced May 20, 2022

Improve azure deploy process (version-control & consolidate config) #2834

Closed

♻ Consolidate php JWT management libs in API app #2835

Open

patcon commented May 21, 2022

View reviewed changes

api/composer.json Outdated Show resolved Hide resolved

patcon added 5 commits May 20, 2022 20:47

Removed dup mention of web-token/jwt-core in composer.json.

c3d9636

Only match mock-auth for specific fake client_id.

537b1c9

Converted windows format file to unix newlines.

b361625

Added editorconfig rules for php indents.

9f8e5a1

More README improvements for auth server. h/t @petertgiles

e96be8f

patcon added 2 commits May 21, 2022 06:24

Merge branch 'main' into mock-oauth2-server-alt

89d8a7d

Removed LocalAuthBearerTokenService and accompanying test.

e9b2569

patcon added 6 commits May 24, 2022 16:20

Removed BearerTokenServiceInterface (maybe phpunit failing).

6484c45

Only try to use Headers apache directive when mod_headers enabled.

f3634b7

Added proxy settings to a new override of default site conf.

fc0dad4

Added comment to explain where mock auth proxy is set up.

cb4f6fd

Got mock-oauth2-server working behind apache proxy.

77a091c

Rename apache site conf file to improve developer comprehension.

1559c58

Small doc fixup. [skip ci]

b50b07d

patcon mentioned this pull request May 25, 2022

Improve e2e test coverage #2671

Closed

17 tasks

Merge branch 'main' into mock-oauth2-server-alt

c628eb8

patcon requested a review from petertgiles May 25, 2022 23:36

patcon mentioned this pull request May 26, 2022

Finish migrating from email to sub for auth #2871

Merged

10 tasks

patcon commented May 26, 2022

View reviewed changes

tristan-orourke approved these changes May 26, 2022

View reviewed changes

petertgiles approved these changes May 27, 2022

View reviewed changes

patcon added 2 commits May 27, 2022 13:10

Added auth folder to gitignore for now.

22a753f

Merge branch 'main' into mock-oauth2-server-alt

27cf309

patcon merged commit e22d90f into main May 27, 2022

patcon deleted the mock-oauth2-server-alt branch May 27, 2022 17:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace laravel auth app with mock-oauth2-server running in docker #2813

Replace laravel auth app with mock-oauth2-server running in docker #2813

patcon commented May 18, 2022 •

edited

Loading

petertgiles commented May 19, 2022 •

edited

Loading

esizer commented May 19, 2022

patcon commented May 19, 2022

patcon commented May 19, 2022 •

edited

Loading

petertgiles left a comment

patcon commented May 20, 2022

patcon commented May 21, 2022

patcon commented May 25, 2022 •

edited

Loading

patcon commented May 25, 2022 •

edited

Loading

patcon commented May 25, 2022

petertgiles commented May 26, 2022

tristan-orourke commented May 26, 2022

tristan-orourke commented May 26, 2022

patcon commented May 26, 2022

patcon May 26, 2022

patcon commented May 26, 2022

tristan-orourke commented May 26, 2022 •

edited

Loading

tristan-orourke commented May 26, 2022

petertgiles left a comment

petertgiles commented May 27, 2022

patcon commented May 27, 2022 •

edited

Loading

Replace laravel auth app with mock-oauth2-server running in docker #2813

Replace laravel auth app with mock-oauth2-server running in docker #2813

Conversation

patcon commented May 18, 2022 • edited Loading

Screencast

Steps to review

To Dos

petertgiles commented May 19, 2022 • edited Loading

esizer commented May 19, 2022

patcon commented May 19, 2022

patcon commented May 19, 2022 • edited Loading

petertgiles left a comment

Choose a reason for hiding this comment

patcon commented May 20, 2022

patcon commented May 21, 2022

patcon commented May 25, 2022 • edited Loading

patcon commented May 25, 2022 • edited Loading

Open Questions

patcon commented May 25, 2022

petertgiles commented May 26, 2022

tristan-orourke commented May 26, 2022

tristan-orourke commented May 26, 2022

patcon commented May 26, 2022

patcon May 26, 2022

Choose a reason for hiding this comment

patcon commented May 26, 2022

tristan-orourke commented May 26, 2022 • edited Loading

tristan-orourke commented May 26, 2022

petertgiles left a comment

Choose a reason for hiding this comment

petertgiles commented May 27, 2022

patcon commented May 27, 2022 • edited Loading

patcon commented May 18, 2022 •

edited

Loading

petertgiles commented May 19, 2022 •

edited

Loading

patcon commented May 19, 2022 •

edited

Loading

patcon commented May 25, 2022 •

edited

Loading

patcon commented May 25, 2022 •

edited

Loading

tristan-orourke commented May 26, 2022 •

edited

Loading

patcon commented May 27, 2022 •

edited

Loading