Using a simple self-signed certificate with native-tls identity #91

ZanderFick · 2022-09-14T18:10:43Z

Initially thought this issue was related to #82 but that does not seem to be the case.

I need to generate a (very) short-lived native_tls::TlsStream for one-shot TLS communication over localhost.

Ideally the application can use a temporary native_tls::Identity built from a self-signed certificate generated using rcgen here's an attempt so far:

// main.rs
fn main() {
    let test_certificate = rcgen::generate_simple_self_signed(["localhost".to_string()]).unwrap();

    let pem = test_certificate.serialize_pem().unwrap();

    let pkey = test_certificate.get_key_pair().serialize_pem();

    let identity = native_tls::Identity::from_pkcs8(pem.as_bytes(), pkey.as_bytes()).unwrap();

    /* 
    Panics:
    Os { code: -2146881269, kind: Uncategorized, message: "ASN1 bad tag value met." }
    */
}

# cargo.toml
# ...
[dependencies]
rcgen = "0.9"
native-tls = "0.2.10"

As indicated, an ASN1 bad tag value met is raised by the OS when trying to use the certificate and private key.

Any ideas on what is going wrong here?

(This is on Windows 10 and 11)

The text was updated successfully, but these errors were encountered:

est31 · 2022-09-14T18:13:59Z

First, what's your OS? And OS version? Does your OS support P-256?

ZanderFick · 2022-09-14T18:15:38Z

First, what's your OS? And OS version? Does your OS support P-256?

Apologies realised I did not add that info initially;

Windows 10 and Windows 11 on two machines, I believe it does support it?

est31 · 2022-09-14T18:27:44Z

Can you try with an existing key, different algorithms, and maybe generally with some variations in which features to enable etc.? Maybe with a little bit of debugging we can find the cause. I don't have access to windows 10 environment atm.

ZanderFick · 2022-09-14T18:30:38Z

Will do and report back!

ZanderFick · 2022-09-14T19:35:26Z

Will do and report back!

Here's what I've tried so far:

All combinations of:

Algorithm Options:

The 6 PKCS options in rcgen

Certificate Authority Options:

rcgen::IsCa::SelfSignedOnly
rcgen::IsCa::Ca(rcgen::BasicConstraints::Unconstrained)

Key Id Method Options:

The three SHA sized in rcgen::KeyIdMethod

The outcomes only vary by Algorithm (regardless of CA and KId options):

algorithm	outcome
PKCS_RSA_SHA256	Certificate::from_params -> KeyGenerationUnavailable
PKCS_RSA_SHA384	Certificate::from_params -> KeyGenerationUnavailable
PKCS_RSA_SHA512	Certificate::from_params -> KeyGenerationUnavailable
PKCS_ECDSA_P256_SHA256	Identity::from_pkcs8s -> { Os: -2146881269, message: "ASN1 bad tag value met."}
PKCS_ECDSA_P384_SHA384	Identity::from_pkcs8s -> { Os: -2146881269, message: "ASN1 bad tag value met."}
PKCS_ED25519	Identity::from_pkcs8s -> { Os: -2146885630, message: "An error occurred during encode or decode operation."}

Does this highlight anything else I can test over?
Or perhaps indicate where I should deep-dive debugging-wise?

Updated cargo.toml:

[dependencies]
rcgen = { version = "0.9.3", features = ["pem",  "x509-parser"]}
native-tls = "0.2.10"
tabled = "0.8.0"

Updated main.rs:

use tabled::{Table, Tabled};

fn make_identity(
    alg: &'static rcgen::SignatureAlgorithm,
    ca_option: rcgen::IsCa,
    kid_option: rcgen::KeyIdMethod,
) -> Result<native_tls::Identity, String> {
    let mut certificate_parameters = rcgen::CertificateParams::new(["localhost".to_string()]);
    certificate_parameters.alg = alg;
    certificate_parameters.is_ca = ca_option;
    certificate_parameters.key_identifier_method = kid_option;

    let test_certificate = rcgen::Certificate::from_params(certificate_parameters)
        .map_err(|err| format!("Certificate::from_params: \n {:#?}", err))?;

    // Serialises the certificate to PKCS#8 PEM
    let pem = test_certificate.serialize_pem().unwrap();

    // Serialises to PKCS#8 format in PEM
    let pkey = test_certificate.get_key_pair().serialize_pem();

    native_tls::Identity::from_pkcs8(pem.as_bytes(), pkey.as_bytes())
        .map_err(|err| format!("Identity::from_pkcs8: \n {:#?}", err))
}

// Output helpers
#[derive(Tabled)]
struct CombinationCheck<'a> {
    algorithm: &'a str,
    ca_option: &'a str,
    key_id_option: &'a str,
    error: String,
}


fn main() {
    // Algorithms
    let algorithms = &[
        (&rcgen::PKCS_RSA_SHA256, "PKCS_RSA_SHA256"),
        (&rcgen::PKCS_RSA_SHA384, "PKCS_RSA_SHA384"),
        (&rcgen::PKCS_RSA_SHA512, "PKCS_RSA_SHA512"),
        (&rcgen::PKCS_ECDSA_P256_SHA256, "PKCS_ECDSA_P256_SHA256"),
        (&rcgen::PKCS_ECDSA_P384_SHA384, "PKCS_ECDSA_P384_SHA384"),
        (&rcgen::PKCS_ED25519, "PKCS_ED25519")
    ];

    // Ca options
    let ca_options = &[
        (rcgen::IsCa::SelfSignedOnly, "IsCa::SelfSignedOnly"),
        (rcgen::IsCa::Ca(rcgen::BasicConstraints::Unconstrained), "IsCa::Ca(rcgen::BasicConstraints::Unconstrained)")
    ];

    // Key Is Method options
    let key_id_options = &[
        (rcgen::KeyIdMethod::Sha256, "Sha256"),
        (rcgen::KeyIdMethod::Sha384, "Sha384"),
        (rcgen::KeyIdMethod::Sha512, "Sha512"),
    ];

    let mut tests = vec![];

    for (alg, name) in algorithms {
        for (ca_option, ca_option_name) in ca_options {
            for (kid_option, kid_name) in key_id_options {
                let mut outcome = CombinationCheck {
                    algorithm: name,
                    ca_option: ca_option_name,
                    key_id_option: kid_name,
                    error: "".to_string(),
                };

                match make_identity(
                    alg,
                    ca_option.clone(),
                    kid_option.clone()) {
                    Err(err_str) => { outcome.error = format!("{} failed: \n {}", name, err_str).to_string() }
                    _ => ()
                };

                tests.push(outcome);
            }
        }
    };

    let table = Table::new(tests).to_string();

    println!("{}", table);
}

ZanderFick · 2022-09-14T20:10:28Z

Comparing certificates:

Generated using openssl:

openssl req -x509 -nodes -days 365 -newkey rsa:512 -keyout selfsigned.key.pem -out selfsigned-x509.crt -subj "/CN=localhost"

selfsigned-x509.crt:

selfsigned.key.pem:

Test:

use std::fs;
use std::io::Read;

fn main() {
    let mut crt = vec![];
    let mut pkey = vec![];

    fs::File::open("selfsigned-x509.crt").unwrap()
        .read_to_end(&mut crt).unwrap();

    fs::File::open("selfsigned.key.pem").unwrap()
        .read_to_end(&mut pkey).unwrap();

    let identity = native_tls::Identity::from_pkcs8(&crt, &pkey).unwrap();
    // Does not panic
}

native_tls::Identity::from_pkcs8 does not panic in this instance

est31 · 2022-09-14T20:13:04Z

Can you try loading the key via rcgen, to test whether key generation is the cause of the issue?

ZanderFick · 2022-09-14T20:44:17Z

Do you mean as follows:

use std::fs;
use std::io::Read;

fn main() {
    let mut pkey = vec![];
    let mut crt = vec![];

    fs::File::open("selfsigned.key.pem").unwrap()
        .read_to_end(&mut pkey).unwrap();
    fs::File::open("selfsigned-x509.crt").unwrap()
        .read_to_end(&mut crt).unwrap();

    let key_pair = rcgen::KeyPair::from_pem(
        &String::from_utf8(pkey).unwrap()
    ).unwrap();
    /* ^This Panics^:
    CouldNotParseKeyPair

    Files are generated using:
    openssl req -x509 -nodes -days 365 -newkey rsa:512 -keyout selfsigned.key.pem -out selfsigned-x509.crt -subj "/CN=localhost" -outform PEM
     */

    let rcgen_cert_params = rcgen::CertificateParams::from_ca_cert_pem(
        &String::from_utf8(crt).unwrap(),
        key_pair
    ).unwrap();

    // Use the rcgen cert
    let test_certificate = rcgen::Certificate::from_params(rcgen_cert_params).unwrap();

    let pem = test_certificate.serialize_pem().unwrap();

    let pkey = test_certificate.get_key_pair().serialize_pem();

    let identity = native_tls::Identity::from_pkcs8(pem.as_bytes(), pkey.as_bytes()).unwrap();
}

est31 · 2022-09-14T21:13:39Z

I think ring does not support RSA-512. The correct creation command is here: https://docs.rs/ring/latest/ring/signature/struct.RsaKeyPair.html#method.from_pkcs8

ZanderFick · 2022-09-15T05:57:04Z

Thanks.

Generated a key with:

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048  -pkeyopt rsa_keygen_pubexp:65537 |  openssl pkcs8 -topk8 -nocrypt -outform pem > private-key.pk8

Generated a Certificate with:

openssl req -x509 -nodes -days 365 -key private-key.pk8 -inform pem -out selfsigned-x509.crt -outform pem

selfsigned-x509.crt

private-key.pk8

Confirmed it still works manually with:

use std::fs;
use std::io::Read;

fn main() {
    let mut pkey = vec![];
    let mut crt = vec![];

    fs::File::open("private-key.pk8").unwrap()
        .read_to_end(&mut pkey).unwrap();
    fs::File::open("selfsigned-x509.crt").unwrap()
        .read_to_end(&mut crt).unwrap();

    let identity = native_tls::Identity::from_pkcs8(&crt, &pkey).unwrap();
}

Trying it with rcgen, it now works!

use std::fs;
use std::io::Read;

fn main() {
    let mut pkey = vec![];
    let mut crt = vec![];

    fs::File::open("private-key.pk8").unwrap()
        .read_to_end(&mut pkey).unwrap();
    fs::File::open("selfsigned-x509.crt").unwrap()
        .read_to_end(&mut crt).unwrap();

    let key_pair = rcgen::KeyPair::from_pem(
        &String::from_utf8(pkey).unwrap()
    ).unwrap();

    let rcgen_cert_params = rcgen::CertificateParams::from_ca_cert_pem(
        &String::from_utf8(crt).unwrap(),
        key_pair
    ).unwrap();

    // Use the rcgen cert
    let test_certificate = rcgen::Certificate::from_params(rcgen_cert_params).unwrap();

    let pem = test_certificate.serialize_pem().unwrap();

    let pkey = test_certificate.get_key_pair().serialize_pem();

    let identity = native_tls::Identity::from_pkcs8(pem.as_bytes(), pkey.as_bytes()).unwrap();
}

Does this mean the problem does indeed lie with the generation?

est31 · 2022-09-15T13:10:09Z

Does this mean the problem does indeed lie with the generation?

It might be with the generation, it might be with RSA. If you are still interested, can try generating an P-256 key using the openssl cli utility, and then importing it similar to above.

ZanderFick · 2022-09-15T17:02:07Z

Does this mean the problem does indeed lie with the generation?

It might be with the generation, it might be with RSA. If you are still interested, can try generating an P-256 key using the openssl cli utility, and then importing it similar to above.

I'll give that a go, thanks!

ZanderFick · 2022-09-15T17:44:39Z

Is this what you meant?
_{(My knowledge of openssl is extremely limited, sorry!)}

openssl ecparam -name prime256v1 -genkey -noout | openssl pkcs8 -topk8 -nocrypt -outform pem > private-key.pk8

openssl req -x509 -nodes -days 365 -key private-key.pk8 -inform pem -out selfsigned-x509.crt -outform pem

private-key.pk8

selfsigned-x509.crt

And then this actually does raise the "ASN1 bad tag value met." error!

use std::fs;
use std::io::Read;

fn main() {
    let mut pkey = vec![];
    let mut crt = vec![];

    fs::File::open("private-key.pk8").unwrap()
        .read_to_end(&mut pkey).unwrap();
    fs::File::open("selfsigned-x509.crt").unwrap()
        .read_to_end(&mut crt).unwrap();

    let identity = native_tls::Identity::from_pkcs8(&crt, &pkey).unwrap();
    /*
        Raises  Os { code: -2146881269, kind: Uncategorized, message: "ASN1 bad tag value met." }
        
        With the openssl p-256 key        
    */
}

Is there a way to configure the generation to get something that will work?

est31 · 2022-09-15T19:13:55Z

And then this actually does raise the "ASN1 bad tag value met." error!

Oh that's good because then we know that this is because native_tls does not support P-256 for some reason, even if openssl is used for the generation of the entire certificate. Maybe the openssl parameters are wrong, maybe there are explanations for the correct parameters on the internet, idk.

Anyways, this doesn't seem like an rcgen bug to me, but I think this discussion has still been very useful for future readers. I'm closing this.

DanielJoyce · 2023-07-07T18:41:40Z

reading linked issues, it appears macs / windows expect pkcs8 format to be encrypted?

jfaust · 2023-07-26T19:03:10Z

It looks like if you explicitly generate the keypair, RSA works on Windows, and is then loadable. The rsa-irc example in this repo does this:

    params.alg = &rcgen::PKCS_RSA_SHA256;

    let mut rng = OsRng;
    let bits = 2048;
    let private_key = RsaPrivateKey::new(&mut rng, bits)?;
    let private_key_der = private_key.to_pkcs8_der()?;
    let key_pair = rcgen::KeyPair::try_from(private_key_der.as_bytes()).unwrap();
    params.key_pair = Some(key_pair);

est31 · 2023-07-27T05:09:53Z

Yeah the issue might be the openssl command. The ring docs have example invocations that generate in the format that ring expects: https://docs.rs/ring/latest/ring/signature/struct.RsaKeyPair.html#method.from_pkcs8

est31 closed this as completed Sep 15, 2022

ZanderFick mentioned this issue Sep 15, 2022

Cannot use self-signed certificate generated with P-256 on Windows sfackler/rust-native-tls#233

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using a simple self-signed certificate with native-tls identity #91

Using a simple self-signed certificate with native-tls identity #91

ZanderFick commented Sep 14, 2022 •

edited

Loading

est31 commented Sep 14, 2022

ZanderFick commented Sep 14, 2022

est31 commented Sep 14, 2022

ZanderFick commented Sep 14, 2022

ZanderFick commented Sep 14, 2022 •

edited

Loading

ZanderFick commented Sep 14, 2022 •

edited

Loading

est31 commented Sep 14, 2022

ZanderFick commented Sep 14, 2022

est31 commented Sep 14, 2022

ZanderFick commented Sep 15, 2022 •

edited

Loading

est31 commented Sep 15, 2022

ZanderFick commented Sep 15, 2022

ZanderFick commented Sep 15, 2022

est31 commented Sep 15, 2022

DanielJoyce commented Jul 7, 2023

jfaust commented Jul 26, 2023 •

edited

Loading

est31 commented Jul 27, 2023

Using a simple self-signed certificate with native-tls identity #91

Using a simple self-signed certificate with native-tls identity #91

Comments

ZanderFick commented Sep 14, 2022 • edited Loading

est31 commented Sep 14, 2022

ZanderFick commented Sep 14, 2022

est31 commented Sep 14, 2022

ZanderFick commented Sep 14, 2022

ZanderFick commented Sep 14, 2022 • edited Loading

ZanderFick commented Sep 14, 2022 • edited Loading

Comparing certificates:

est31 commented Sep 14, 2022

ZanderFick commented Sep 14, 2022

est31 commented Sep 14, 2022

ZanderFick commented Sep 15, 2022 • edited Loading

est31 commented Sep 15, 2022

ZanderFick commented Sep 15, 2022

ZanderFick commented Sep 15, 2022

est31 commented Sep 15, 2022

DanielJoyce commented Jul 7, 2023

jfaust commented Jul 26, 2023 • edited Loading

est31 commented Jul 27, 2023

ZanderFick commented Sep 14, 2022 •

edited

Loading

ZanderFick commented Sep 14, 2022 •

edited

Loading

ZanderFick commented Sep 14, 2022 •

edited

Loading

ZanderFick commented Sep 15, 2022 •

edited

Loading

jfaust commented Jul 26, 2023 •

edited

Loading