Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Self Signed Certs preventing Login #320

Open
Jaffythethird opened this issue Aug 8, 2022 · 36 comments · May be fixed by #397
Open

Self Signed Certs preventing Login #320

Jaffythethird opened this issue Aug 8, 2022 · 36 comments · May be fixed by #397
Labels
bug Something isn't working

Comments

@Jaffythethird
Copy link

Summary
I'm on a company VPN, trying to make a leapp integration for a customer environment. Using the browser, I can sign in (it requires having https in the URL, but it returns as "insecure" because of the self signed cert) through the AWS SSO. However, using the Leapp integration causes nothing to pop up, even when browser is selected. Using the CLI, I can now see why that is. Error code is written below.

Version
Leapp Version 0.13.4

Expected Response:
I understand why insecure connections may be blocked, but it seems to fail without displaying an error to the user (unless using the CLI), and it would be nice to have an --insecure override if there's not other way of resolving this. How might I get this to connect for now?

CLI Response (leapp integration login):

node:events:491
    throw er; Unhandled 'error' event
    ^
{
Error: self signed certificate in certificate chain
    at TLSSocket.onConnectSecure (node:_tls_wrap:1534:34)
    at TLSSocket.emit (node:events:513:28)
    at TLSSocket.emit (node:events:489:12)
    at TLSSocket._finishInit (node:_tls_wrap:948:8)
    at ssl.onhandshakedone (node:_tls_wrap:729:12)
Emitted 'error' event on Writable instance at:
    at eventHandlers.<computed> (/usr/local/lib/node_modules/@noovolari/leapp-cli/node_modules/follow-redirects/index.js:14:24)
    at ClientRequest.emit (node:events:513:28)
    at ClientRequest.emit (node:events:489:12)
    at TLSSocket.socketErrorListener (node:_http_client:481:9)
    at TLSSocket.emit (node:events:513:28)
    at TLSSocket.emit (node:domain:489:12)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
  code: 'SELF_SIGNED_CERT_IN_CHAIN'
}

Node.js v18.7.0

Desktop:

  • OS: MacOS Monterey
  • OS Version 12.5
  • Leapp Version 0.13.4
@Jaffythethird Jaffythethird added the bug Something isn't working label Aug 8, 2022
@ghost
Copy link

ghost commented Aug 9, 2022

Good catch!

What happens if you add the self-signed cert or the CA it was created with into the Keychain as trusted CA?

@urz9999
Copy link
Contributor

urz9999 commented Aug 9, 2022

Good catch indeed! The problem is that this error was not watched correctly, so there is no proper error handling for it at the moment, nonetheless will get a look into it, as we are planning another release soon

@Jaffythethird
Copy link
Author

What happens if you add the self-signed cert or the CA it was created with into the Keychain as trusted CA?

Sorry for the late response, but I've been trying things like that. None of them worked for me, though I am admittedly not the most experienced at Keychain/Cert management. Nonetheless, I'm pretty sure that the several ways I went about adding them into the Keychain as trusted included a correct way of doing it. No fix, though.

The problem is that this error was not watched correctly, so there is no proper error handling for it at the moment, nonetheless will get a look into it, as we are planning another release soon

Do you think that the error handling will allow working around this problem, or just making sure that the user is notified as to why they can't get their login to start?

@ghost
Copy link

ghost commented Aug 12, 2022

Sorry for the late response, but I've been trying things like that. None of them worked for me, though I am admittedly not the most experienced at Keychain/Cert management. Nonetheless, I'm pretty sure that the several ways I went about adding them into the Keychain as trusted included a correct way of doing it. No fix, though.

Thank you for testing. I can try to replicate the issue and work this out by creating similar AWS SSO environment unless @urz9999 is already on it?

Do you think that the error handling will allow working around this problem, or just making sure that the user is notified as to why they can't get their login to start?

A bit of both, depending how this issue will get fixed. I was thinking instructing the user to for example turning "allow insecure connection" (not yet implemented) setting on or assist with marking certificate as trusted. It should work also as a warning for possible man-in-the-middle attack if user suddenly starts seeing the error with previously working connection.

@urz9999
Copy link
Contributor

urz9999 commented Aug 12, 2022

Actually, you are free to try it by yourself currently, I'm on a new feature that hopefully will be released next week

@ghost
Copy link

ghost commented Aug 12, 2022

It seems that Node uses it's own CA store and that's why fiddling with Key Chain didn't work out.
I'll add possibility to import own CA/cert, option to connect without certificate checking and add exception handling.

@tgjohnst
Copy link

tgjohnst commented Sep 2, 2022

Chiming in to say I'm testing a VPN+firewall solution with SSL inspection and am also affected by this issue.

AWSCLI had a similar issue because it uses the certs vendored by botocore, rather than trusting the system keychain. That was fixed by either appending the custom certs we use to the botocore bundle or pointing AWSCLI to a custom bundle via either the ca_bundle var in .aws/config or the AWS_CA_BUNDLE environment variable.

The ability to provide a custom cert bundle (we have at least 2 certs that need trusting), add trusted certs to whichever bundle Leapp is using, disable SSL validation, or at the very least a place where Leapp's cert bundle lives that's editable would be extremely helpful. Is there currently a timeline for implementation of this, or a feature branch open? Or any workaround that wouldn't involve building Leapp from source?

@ghost
Copy link

ghost commented Sep 2, 2022

Thank you for chiming in.

I have a working fix for this and are working on dialogs to detect and import required certs.
I can probably in few days create a pr for a version with basic import functionality and if it goes smoothly, also with a checkbox to disable ssl validation for the session. It's not going to be a global option, at least yet.

By two different certs, do you mean a cert-chain with CA and intermediate certs or just a cert-bundle with two different certs and if latter, do you need to have both certs for one session or one cert for few sessions and other for another sessions?

I mean is it enough for now if you can assign one cert for a specific IAM User or IAM Federated-session or AWS SSO integration or do you have a need for two different trusted certs to trust for one session or SSO integration?

Maybe I should also implement an application wide setting for using proxy.

But first things first. I'll create a PR for ability to import cert or cert-bundle for a session.

@tgjohnst
Copy link

tgjohnst commented Sep 7, 2022

@c-sami Thank you, and great questions. I'm relatively new to cert management and working on grokking all that's involved here but am happy to test/review/assist with the PR whenever it's ready!

We have an organization-wide firewall/VPN doing SSL inspection with it's own root cert. By two different certs, I mean a cert chain with CA + intermediate, and these would be needed for all sessions when on VPN, they're not session-specific in my context. The ability to have a trusted cert [bundle] for a session or disable SSL validation for a session would certainly be a valid solution.

@ghost
Copy link

ghost commented Sep 12, 2022

Hi,

I built an UNOFFICIAL build for testing skipping certificate validation.
Version has UI for adding custom ca certificates, but it is disabled as the implementation is work-in-progress.

https://github.com/siipimutteri/siipimutteri-leapp/releases (My own UNOFFICIAL build. Not related to Noovolari. Use at your own risk! Will remove soon. ONLY FOR TESTING THIS ISSUE)
Built from: https://github.com/siipimutteri/siipimutteri-leapp/tree/1a4d4f7a4d64920f2cade404166171d6b1d0009e
Build job: https://github.com/siipimutteri/siipimutteri-leapp/actions/runs/3029033658

@ghost
Copy link

ghost commented Sep 12, 2022

This is the proposal for a new "Network"-tab that can be accessed from the upper-right "options"-button.
Depending how much there is space, I could also move proxy-settings under it.
Final version does not have so much text and the warning-disclaimer will be accessible under tooltip.

But for this testing build, I wanted to leave the disclaimer clearly visible.

image

@andreacavagna01
Copy link
Contributor

Let me know if that build solves the problem. At this time, I'm unable to test it, then we will work to add a solution in the app in the next releases

@ghost
Copy link

ghost commented Sep 12, 2022

I tested it just by creating own CA, intermediate CA and server cert, firing up nginx docker container with created server certificate and pointed AWS SSO integration into it. Without the skip-option, I got SSL error and with it login error.

And that reminded me that I forgot to cover IAM-users! Maybe I'll be able to push a new test release even today. And use 0.14.2 as a base. Now it was based on 0.14.1.

At least for Windows, it is possible to skip installation, unpack the zip archives and just start the Leapp from the downloaded zip.

@tgjohnst
Copy link

tgjohnst commented Sep 12, 2022

Wow, thank you for the quick turnaround!

I downloaded the test build and was able to test. Unfortunately was unable to test on my machine as the network tab was incomplete and the user defined CA field was unresponsive/uninteractible when clicked. Not sure what would cause this. One guess - I'm running on a Macbook Pro M1, is the test build for macOS targeted at ARM64?

edit - missed your note about the custom cert box being disabled. Although my certificate validation checkbox was missing the warning text in your screenshot, can confirm that the test build does successfully restore function on VPN (where we have MITM happening) with the skip validation option enabled! I did notice the build was a lot slower in general to start and interact with, but guessing that's not due to this change in particular.

Tested on MacOS 12.5.1, ARM64 (M1), with an AWS SSO session

@ghost
Copy link

ghost commented Oct 4, 2022

Sorry about the delay. I'm implementing this with a best-effort basis and are currently resolving issues with all other connections in addition to sso sessions. It turned out that there are multiple places that needs same certificates and maybe system certificates could be used. It needs at least three different implementations. One for each supported OS. Testing and validating has been a bit problematic even with using mitmproxy.

@ghost
Copy link

ghost commented Nov 9, 2022

Still working on this. Slowly though because lack of time.
Experimenting if win-ca, mac-ca and linux-ca or system-ca would work or is own implementation the only way.

@tgjohnst
Copy link

tgjohnst commented Dec 1, 2022

Thank you for the updates, @c-sami

@siipimutteri
Copy link

siipimutteri commented Dec 12, 2022

c-sami "ghost" here. I just changed from my work account to personal one.
I have been fiddling around with this using transparent mitmproxy and I think I got it finally working by using 'system-ca'.
Only troubles are with windows-build (damned node-gyp) and cli.
I tested mac-ca that hung the whole app, built my own cert-sync-module with rust and lots of different things.
But anyhow.. now we are getting close.

Here's current setup: siipimutteri@277325a
Just ignore .github workflow file and concentrate on system-ca and systemCertsAsync lines.

edit: Here's clean version: siipimutteri@4ac16ee
Funny thing is that at one point I had hundreds lines of code and now only few lines that are actually what seemed to be needed.

I'll publish linux and mac (intel & arm) binaries for testing.
Windows build will be available after I have figured out what's the issue with rebuilding node-gyp.

@siipimutteri
Copy link

siipimutteri commented Dec 13, 2022

Unofficial builds for testing: https://github.com/siipimutteri/siipimutteri-leapp/releases/tag/0.16.2-126cf39.1-320
Cli not working.
No Windows build yet.

I have tested only Mac Arm64 build mac x64 and arm64 builds.

To run in mac, you'll need to bypass security warning with ctrl+click as I don't have Apple Developer subscription and didn't sign the binary.

@siipimutteri
Copy link

Tested with Mac x64 and it works.
Noticed that we don't have win32 build, but not sure if we need one.

Testing with Linux and Windows is still todo.
Catching the cert error is still an issue. Leapp integration login stops silently working if there is a certificate error and Leapp needs to be restarted.

@siipimutteri
Copy link

Windows 64bit build is also available. Node-gyp issue is strange, but got around it by using npm ci instead of npm install

@tgjohnst
Copy link

Can confirm that build Leapp-0.16.2-126cf39.1-mac-x64 works and successfully gets around the certificate issue while logged in on VPN on my M1 macbook running macOS Ventura. I don't have a Windows or Linux machine on our VPN so am unable to test in either of those environments. Any other updates to this effort?

@ericvilla
Copy link
Contributor

Firstly, thank you @siipimutteri for the commitment!
Given @tgjohnst feedback, the only OS that's still not tested is Linux. I think we can merge the proposed Pull Request into the main branch, test it, and release it during the next week.

@ericvilla ericvilla reopened this Feb 24, 2023
@siipimutteri
Copy link

There are still few issues about handling SSL and connection error events that I have been trying to debug and sort out.
For being able to release the long awaited feature, I'll create an opt-in setting for enabling "Trust system CA (beta)".
I think I can do the PR for it this weekend.

Currently GUI works but CLI doesn't, when using mitmproxy and trusting mitmproxy's CA. It might be a mitmproxy issue or a wider one. Maybe @tgjohnst can test the CLI after the release.

@siipimutteri siipimutteri linked a pull request Feb 25, 2023 that will close this issue
@ericvilla
Copy link
Contributor

Can confirm that build Leapp-0.16.2-126cf39.1-mac-x64 works and successfully gets around the certificate issue while logged in on VPN on my M1 macbook running macOS Ventura. I don't have a Windows or Linux machine on our VPN so am unable to test in either of those environments. Any other updates to this effort?

@tgjohnst once logged into AWS SSO, are you able to start Leapp AWS SSO Role Sessions via VPN?

@siipimutteri
Copy link

Figured out and fixed the issue with the CLI. Now AWS SSO Integration login works with desktop app and cli.
I replaced wireguard&mitmproxy with pfSense Plus, OpenVPN and Squid to get more authentic environment.
Next step is to fully test AWS SSO integration and then implement feature with other access methods, integrations and update notifier.

@ericvilla
Copy link
Contributor

@Jaffythethird we're currently working with Sami to introduce this feature in the next release.

I'm wondering what are your configuration details: is the customer AWS environment limiting access to AWS SSO APIs to a specific range of sourceIp addresses? In this case, I'm quite sure it applies to all AWS SDK API calls.
Is the VPN server deployed in the customer environment?

@Jaffythethird
Copy link
Author

Admittedly, I've gotten into a workflow that doesn't require leapp. That being said, I appreciate all the work being put into this.

And, as for the customer environment... I've not been with them for a while and can't say for sure what the configuration was. I will say, since it deals with compliance stuff, it was probably delegated to FIPS endpoints

@tgjohnst
Copy link

Can confirm that build Leapp-0.16.2-126cf39.1-mac-x64 works and successfully gets around the certificate issue while logged in on VPN on my M1 macbook running macOS Ventura. I don't have a Windows or Linux machine on our VPN so am unable to test in either of those environments. Any other updates to this effort?

@tgjohnst once logged into AWS SSO, are you able to start Leapp AWS SSO Role Sessions via VPN?

sorry for the late reply, missed the ping. Yes, to answer in further detail:

  • realized that the last time, I was testing with the x64 build even on an arm64 system. I tried the arm64 version and macOS reported that it's broken, so something is wrong with the arm64 build for this release (Leapp-0.16.2-126cf39.1-mac-arm64). Proceeded testing with -x64
  • In the main release branch, while on VPN, the process of logging into an SSO integration and starting an SSO session would hang indefinitely without popping up a browser window. Can confirm that starting Leapp build Leapp-0.16.2-126cf39.1-mac-x64, connecting to the VPN, and logging into an AWS SSO integration + starting a session works correctly on the x64 build via GUI. Screenshot attached
image

Any updates on the developer side?

@siipimutteri
Copy link

@ericvilla has inspected the issue and found out that fixing the issue for all session types needs a lot of work. I haven’t done any action for a while, but now that you pinged, I see a need for the fix and maybe we could introduce the existing fix for the sso integration only and fix other session types later? I believe fix for sso sessions is almost ready for release.

I can start working with fixing other session types as I have now all supported session types at my hand, including federation via GoogleWorkspaces (GApps).

@tgjohnst
Copy link

Sami, thanks again for your work here - any updates on this? The fix for SSO sessions would be sufficient for us, and we're facing an impending always-on VPN+SSL inspection rollout at my company in the next couple months.

@siipimutteri
Copy link

I'll update the codebase and do a PR for supporting self-signed cert with SSO only. Maybe even this Sunday, but cannot promise.
I have now Intel and M1 Macs so I can test them, but I need help for testing with Windows and Linux.

@tgjohnst
Copy link

Awesome, @siipimutteri !

What kind of testing would you need done on Windows and Linux? I may be able to spin up some internal VMs to test.

@dalewest
Copy link

@siipimutteri Has there been any progress on this issue? I would be happy to help test as well.

@ollaw
Copy link

ollaw commented Jan 3, 2024

Hi, are there any updates on this ?

@happyhil
Copy link

Got it working on my end by adding the certificate to NODE_EXTRA_CA_CERTS, following by leapp integration login in terminal. However, adding the certificate via the network tap would be very cool in order to make the sso login via app working again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants