HTTP SSL error: "Unknown CA"—going down a Rabbit hole. Maybe an asdf Erlang issue?

[dogweather]

I get this error with both :httpc and HTTPoison.get. Here’s HTTPoison:

iex(1)> HTTPoison.get! "https://grad.tamu.edu/" 22:15:54.179 [notice] TLS :client: In state :certify at ssl_handshake.erl:2138 generated CLIENT ALERT: Fatal - Unknown CA ** (HTTPoison.Error) {:tls_alert, {:unknown_ca, ~c"TLS client: In state certify at ssl_handshake.erl:2138 generated CLIENT ALERT: Fatal - Unknown CA\n"}} (httpoison 2.2.0) lib/httpoison/base.ex:451: HTTPoison.request!/5 iex:1: (file) 

Same with this:

iex(1)> :httpc.request(:get, {'https://grad.tamu.edu/', []}, [], []) 22:24:10.836 [notice] TLS :client: In state :certify at ssl_handshake.erl:2138 generated CLIENT ALERT: Fatal - Unknown CA {:error, {:failed_connect, [ {:to_address, {~c"grad.tamu.edu", 443}}, {:inet, [:inet], {:tls_alert, {:unknown_ca, ~c"TLS client: In state certify at ssl_handshake.erl:2138 generated CLIENT ALERT: Fatal - Unknown CA\n"}}} ]}} 

.tool-versions:

elixir 1.15.2-otp-26 erlang 26.0.2 

I’ve seen this pop up periodically over the years. What’s the root cause?

I’m thinking of making an elixir lib that simply shells out to curl, which has no problems:

$ curl --head https://grad.tamu.edu/ HTTP/2 200 cache-control: private content-length: 46798 content-type: text/html; charset=utf-8 server: Microsoft-IIS/10.0 x-aspnetmvc-version: 5.2 x-aspnet-version: 4.0.30319 x-powered-by: ASP.NET date: Wed, 15 Nov 2023 05:20:20 GMT strict-transport-security: max-age=4294967294 

EDIT: I made this dead simple Hex Package.

CurlEx.get!(url) 

[D4no0]

It seems that the underlying issue is with the certificate chain. You can take a look at this report.

From ssl erlang side, from which all libraries end up doing the request, it seems that erlang is failing to decode the certificate, cases being either the format being bleeding edge or a incorrect ASN.1 definition, which might be the case here.

I would recommend contacting one of their site administrators and inquiring about that, as it seems that this certificate is used for all of their subdomain sites and most probably was generated manually. This is a potential security threat and should be handled on their side ideally instead of using a workaround in the code.

I will do a investigation on why this happens and return here with a more detailed response.

[voltone]

Looks like the server only sends its own certificate, not any intermediary CAs that would allow it to be traced to a trusted root CA. Instead it relies on the client to fetch the intermediate CA(s) from the URL in the Authority Information Access extension. This is supported by browsers and by some CLI tools, but not by Erlang’s ssl. I suspect they would have issues with other non-browser clients too.

For maximum compatibility they should consider adding the trust chain to their server, so it sends all the intermediate CA certificates and the client can complete the trust chain locally without additional network requests.

[D4no0]

I’m not sure about the implementation, however I’ve downloaded the certificate locally and tried to decode it with :public_key.pkix_decode_cert/2 and it failed.

Another issue is for example when using a custom verify_fun, no peer certificate is returned (even though it should be, as you are validating the certificate and chain manually) only {:error, :unknown_ca}

[voltone]

I’m not having any issues decoding the server’s certificate using public_key. The only reason ssl rejects it is because it can’t find that certificate’s issuer (/C=US/O=Internet2/CN=InCommon RSA Server CA 2).

If that certificate were included in the chain sent by the server, then the chain to /C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority could be completed and the certificate verification would succeed.

[D4no0]

Strange, I will try it again later, are you using openssl v1 or v3?

[voltone]

Does not matter: OpenSSL is not used by public_key for decoding certificates.

Here is the PEM encoded server certificate, for reference:

-----BEGIN CERTIFICATE----- MIIPnjCCDgagAwIBAgIRALFxge5FKu03igK//v8wV1owDQYJKoZIhvcNAQEMBQAw RDELMAkGA1UEBhMCVVMxEjAQBgNVBAoTCUludGVybmV0MjEhMB8GA1UEAxMYSW5D b21tb24gUlNBIFNlcnZlciBDQSAyMB4XDTIzMTEwOTAwMDAwMFoXDTI0MTEwODIz NTk1OVowXzELMAkGA1UEBhMCVVMxDjAMBgNVBAgTBVRleGFzMR8wHQYDVQQKDBZU ZXhhcyBBICYgTSBVbml2ZXJzaXR5MR8wHQYDVQQDExZ3ZWItY21zLWxiLmFzLnRh bXUuZWR1MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEArOM6EOdBPsE0 agjTJmBIWaWWFWsRhDDVJOvASSR4p6bcCjgWSgkdm+TLFUEbfAsfvgs3hJu0TY56 1qC5r+YYSWhar+0BK1Fc065jsBiZc1OSeGp5e0tAkfS+CcGGDzCdTgXVzbTOlwMC FU9zQnbKxEyP2MQe1yTp4ohAD86EVXmmnwwuWIketm2CYMbcMGvIq9yPpMuJasGh hCFuJj/k3ogZd5fIDQS0ZK/1WpiaxNK6LEQ8zZjG1TlYgtYAfN5kUSGkJbtrnaiP A0cP+6fNxekB2tlyr5RzDjl0fgblBLDuVVTglVdJiWtkruFMbVkOH1jC7ztIGcNP aCjLWBHhc9gbGSO04SgapOiLDMB5ZhSab/SVK6RBYVEhP9dte11v+0WgLPdqXwfF Z8qOJ/YxltTGkuUEViM58NiYXvIpUpwVTA6bVLyhPpB2W9nSfS6Zi8a0dukU+gHh p+v7TLXQc++V5VFtIsiBgeTsMnS1KZvSifCozHgs3We5ow+UETSyF8pPZcLWXfbY z+j927T+xXF4MxJL03o6nfL9bneAE7SIUtLD9rkymzyQF99hmi+SzYrt/FAVTNJ+ FxaPoBokkd9Cn0Y3vyftq7TauvF5BzxKov7kaAm9sfjgAYDbpk0SNCYKEMhS0Q65 nl66mzCMojuw8LDXw0HTP5Dwzs+ukpcCAwEAAaOCCu4wggrqMB8GA1UdIwQYMBaA FO9MAJKm+3YuXpXiyV+HGxnVTeLZMB0GA1UdDgQWBBTVR/Y7TXwSVdbwJAO08keX O8daDDAOBgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggr BgEFBQcDAQYIKwYBBQUHAwIwSQYDVR0gBEIwQDA0BgsrBgEEAbIxAQICZzAlMCMG CCsGAQUFBwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzAIBgZngQwBAgIwQAYD VR0fBDkwNzA1oDOgMYYvaHR0cDovL2NybC5zZWN0aWdvLmNvbS9JbkNvbW1vblJT QVNlcnZlckNBMi5jcmwwcAYIKwYBBQUHAQEEZDBiMDsGCCsGAQUFBzAChi9odHRw Oi8vY3J0LnNlY3RpZ28uY29tL0luQ29tbW9uUlNBU2VydmVyQ0EyLmNydDAjBggr BgEFBQcwAYYXaHR0cDovL29jc3Auc2VjdGlnby5jb20wggF8BgorBgEEAdZ5AgQC BIIBbASCAWgBZgB1AHb/iD8KtvuVUcJhzPWHujS0pM27KdxoQgqf5mdMWjp0AAAB i7ZAX3wAAAQDAEYwRAIgcw+Km0Lp0M5Ys2W0Dz0EHp2tEQhrSHg0YnK0x07O0CcC IGuBz8dOXJdSVch1eXpHT57V3xGsrthTfyYKEGO9W0sCAHUAPxdLT9ciR1iUHWUc hL4NEu2QN38fhWrrwb8ohez4ZG4AAAGLtkBgFgAABAMARjBEAiAsO8+RgpHTdK5G ppRR7W5XfoFQdNB0Vr5jwVXGQ7RagwIgI+J7UvPXYBfYbTevcVxkiE4eOyW+Bo2C nEVIsQIL0gIAdgDuzdBk1dsazsVct520zROiModGfLzs3sNRSFlGcR+1mwAAAYu2 QF+zAAAEAwBHMEUCIQCaqIv6Y4k21dKTJOIWrp/1bLl0IoDHKR9/EUgBUqjBNQIg KR1cNczB1RE0kEAG6gHDwDVKU5g9uhWlOg0u65dFywAwggfqBgNVHREEggfhMIIH 3YIWd2ViLWNtcy1sYi5hcy50YW11LmVkdYINYWJwYS50YW11LmVkdYITYWRtaW4u YWJwYS50YW11LmVkdYIdYWRtaW4uYWNjb3VudGFiaWxpdHkudGFtdS5lZHWCGWFk bWluLmFkbWlzc2lvbnMudGFtdS5lZHWCGWFkbWluLmFnZXAtdHhhcm0udGFtdS5l ZHWCFGFkbWluLmFnZ2llLnRhbXUuZWR1ghJhZG1pbi5hc2MudGFtdS5lZHWCGWFk bWluLmFzc2Vzc21lbnQudGFtdS5lZHWCE2FkbWluLmF2cGEudGFtdS5lZHWCG2Fk bWluLmNhcmVlcmNlbnRlci50YW11LmVkdYIUYWRtaW4uY2lydGwudGFtdS5lZHWC F2FkbWluLmNwZWRpbmZvLnRhbXUuZWR1ghJhZG1pbi5jdGUudGFtdS5lZHWCF2Fk bWluLmRpc3RhbmNlLnRhbXUuZWR1ghhhZG1pbi5kaXZlcnNpdHkudGFtdS5lZHWC HGFkbWluLmZhY3VsdHlzZW5hdGUudGFtdS5lZHWCG2FkbWluLmZpbmFuY2lhbGFp ZC50YW11LmVkdYIVYWRtaW4uZ2xvYmFsLnRhbXUuZWR1ghNhZG1pbi5ncmFkLnRh bXUuZWR1ghVhZG1pbi5sYXVuY2gudGFtdS5lZHWCEmFkbWluLmxtcy50YW11LmVk dYIUYWRtaW4ubWJpb3QudGFtdS5lZHWCEmFkbWluLm1sYy50YW11LmVkdYIdYWRt aW4ubXZjY29udGVudDEuYXMudGFtdS5lZHWCHWFkbWluLm12Y2NvbnRlbnQyLmFz LnRhbXUuZWR1gh1hZG1pbi5tdmNjb250ZW50My5hcy50YW11LmVkdYIdYWRtaW4u bXZjY29udGVudDQuYXMudGFtdS5lZHWCHWFkbWluLm12Y2NvbnRlbnQ1LmFzLnRh bXUuZWR1ghthZG1pbi5tdmNzdGFnZTEuYXMudGFtdS5lZHWCG2FkbWluLm12Y3N0 YWdlMi5hcy50YW11LmVkdYIbYWRtaW4ubXZjc3RhZ2UzLmFzLnRhbXUuZWR1ghth ZG1pbi5tdmNzdGFnZTQuYXMudGFtdS5lZHWCG2FkbWluLm12Y3N0YWdlNS5hcy50 YW11LmVkdYIUYWRtaW4ub2FicGEudGFtdS5lZHWCF2FkbWluLnBhbmRlbWljLnRh bXUuZWR1ghlhZG1pbi5wa3B0YW11MDUzLnRhbXUuZWR1ghZhZG1pbi5wcm92b3N0 LnRhbXUuZWR1ghhhZG1pbi5yZWdpc3RyYXIudGFtdS5lZHWCHWFkbWluLnN0dWRl bnRzdWNjZXNzLnRhbXUuZWR1ghdhZG1pbi5zdHVkeWh1Yi50YW11LmVkdYIZYWRt aW4udGFtdXMtYWdlcC50YW11LmVkdYIWYWRtaW4udGVzdGluZy50YW11LmVkdYIT YWRtaW4udHRsYy50YW11LmVkdYIRYWRtaW4udXMudGFtdS5lZHWCEmFkbWluLnV3 Yy50YW11LmVkdYIUYWRtaW4udmlzaXQudGFtdS5lZHWCE2FkbWluLnZtbGMudGFt dS5lZHWCE2FkbWlzc2lvbnMudGFtdS5lZHWCDmFnZ2llLnRhbXUuZWR1ghVhZ2dp ZW9uZXN0b3AudGFtdS5lZHWCD2FnZ2llcy50YW11LmVkdYIMYXNjLnRhbXUuZWR1 ghVjYXJlZXJjZW50ZXIudGFtdS5lZHWCHGNvbnRpbnVpbmdlZHVjYXRpb24udGFt dS5lZHWCEWNwZWRpbmZvLnRhbXUuZWR1ggxjdGUudGFtdS5lZHWCFmZhY3VsdHlz ZW5hdGUudGFtdS5lZHWCFWZpbmFuY2lhbGFpZC50YW11LmVkdYIPZ2xvYmFsLnRh bXUuZWR1gg1ncmFkLnRhbXUuZWR1gg9sYXVuY2gudGFtdS5lZHWCE21hdGhjZW50 ZXIudGFtdS5lZHWCDm1iaW90LnRhbXUuZWR1ggxtbGMudGFtdS5lZHWCF212Y2Nv bnRlbnQxLmFzLnRhbXUuZWR1ghdtdmNjb250ZW50Mi5hcy50YW11LmVkdYIXbXZj Y29udGVudDMuYXMudGFtdS5lZHWCF212Y2NvbnRlbnQ0LmFzLnRhbXUuZWR1ghdt dmNjb250ZW50NS5hcy50YW11LmVkdYIVbXZjc3RhZ2UxLmFzLnRhbXUuZWR1ghVt dmNzdGFnZTIuYXMudGFtdS5lZHWCFW12Y3N0YWdlMy5hcy50YW11LmVkdYIVbXZj c3RhZ2U0LmFzLnRhbXUuZWR1ghVtdmNzdGFnZTUuYXMudGFtdS5lZHWCFnBpdG9j ZG5jc3MuYXMudGFtdS5lZHWCGHBpdG9jZG5tZWRpYS5hcy50YW11LmVkdYIacGl0 b2NkbnNjcmlwdHMuYXMudGFtdS5lZHWCE3BrcHRhbXUwNTMudGFtdS5lZHWCEnJl Z2lzdHJhci50YW11LmVkdYIRc3R1ZHlodWIudGFtdS5lZHWCE3RhbXVzLWFnZXAu dGFtdS5lZHWCFnRlc3RjbXNpdGUuYXMudGFtdS5lZHWCEHRlc3RpbmcudGFtdS5l ZHWCDXR0bGMudGFtdS5lZHWCC3VzLnRhbXUuZWR1gg52aXNpdC50YW11LmVkdTAN BgkqhkiG9w0BAQwFAAOCAYEAFWtuggQYnTnV0GUK/SrRvGkQk+GEdIEGulnp+sVw x2cszdDPG7kJDr70RTrTFgAT72XCxM7FY4/UDducSDLm0x791tNMZXA7jpgfHYe6 mWesw+CEtGzT55/qPKGnyTyjex4RcPwC8juiL6QHr6WiVyvbDlhqdwkZpz1Qy/Um GTXYPO9VF3opI8BnXF9/32C+mlKZRawVl5OFlLJvbZA70hxlUsH7x9OX79i7eGCL yq0rbJJyfhsfoIFfSzN6yaPb3o5j30rTaCZz6+QqUczlOWTRLKibi1daQFGOmADQ VKKySTN+TR7j+tvvH4yUi8Vis8+/vGbvUyEyiZCeaFWTtufB8ck3CNEBV46NxRzo 1DkMheFCiK23ydIHdOlKDauW98Yf1kprp6Ts2uS8v5YXE3nuVJQ5q5qdkS+xQpB6 V+aKGQTYXX4C8YC9hte3V7a2jX8vdk6hfTKT6nKidwcSLGhUxqo7F5YGotFGrj2/ zZEn2+h9sXdhBWahyIpDofL8 -----END CERTIFICATE----- 

[D4no0]

I can confirm that this doesn’t work:

** (MatchError) no match of right hand side value: {:error, {:asn1, {{:invalid_value, 3}, [{:asn1rt_nif, :decode_ber_tlv, 1, [file: ~c"asn1rt_nif.erl", line: 93]}, {:“OTP-PUB-KEY”, :decode, 2, [file: ~c"…/src/OTP-PUB-KEY.erl", line: 1233]}, {:pubkey_cert_records, :decode_cert, 1, [file: ~c"pubkey_cert_records.erl", line: 40]}, {:public_key, :pkix_decode_cert, 2, [file: ~c"public_key.erl", line: 522]}, {:elixir, :“-eval_external_handler/1-fun-2-”, 4, [file: ~c"src/elixir.erl", line: 376]}, {:erl_eval, :do_apply, 7, [file: ~c"erl_eval.erl", line: 750]}, {:elixir, :eval_forms, 4, [file: ~c"src/elixir.erl", line: 361]}, {Module.ParallelChecker, :verify, 1, [file: ~c"lib/module/parallel_checker.ex", line: 112]}]}}}
public_key.erl:526: :public_key.pkix_decode_cert/2
iex:2: (file)

I am using erlang 26.1.2, tried both with otp and plain options.

[voltone]

I think you may be trying to call :public_key.pkix_decode_cert on a PEM encoded certificate, but it expect a DER binary. Try one of these:

pem |> :public_key.pem_decode() |> hd() |> elem(1) |> :public_key.pkix_decode_cert(:otp) pem |> :public_key.pem_decode() |> hd() |> :public_key.pem_entry_decode() 

Or try x509 | Hex for a convenient high-level API for working with certificates and other PKI data structures…

[D4no0]

I think you may be trying to call :public_key.pkix_decode_cert on a PEM encoded certificate

That was exactly what I was doing, thanks!

@voltone

Or try x509 | Hex for a convenient high-level API for working with certificates and other PKI data structures…

Very handy library, I will use it for sure in the future, thanks!

[dogweather]

I chatted with Texas A&M IT helpdesk. They said they’ll send the report on. We’ll see!

[dch]

a few handy tricks for next time:

> curl -v --cert-status https://grad.tamu.edu/ * Trying 128.194.14.62:443... * Connected to grad.tamu.edu (128.194.14.62) port 443 * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (OUT), TLS alert, handshake failure (552): * OpenSSL/3.0.12: error:0A000152:SSL routines::unsafe legacy renegotiation disabled * Closing connection curl: (35) OpenSSL/3.0.12: error:0A000152:SSL routines::unsafe legacy renegotiation disabled 

TLDR, their system is broken. Your client tries to negotiate downwards until it finds a common TLS standard that it can work with, but eventually gives up, as TLS<1.1 is defacto broken and is generally excluded across the public internet now, if configured correctly.

But why?

> openssl s_client -showcerts -connect grad.tamu.edu:443 CONNECTED(00000003) 0020014A48190000:error:0A000152:SSL routines:final_renegotiate:unsafe legacy renegotiation disabled:/usr/src/crypto/openssl/ssl/statem/extensions.c:894: --- no peer certificate available --- No client certificate CA names sent --- SSL handshake has read 7297 bytes and written 326 bytes Verification: OK --- New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : 0000 Session-ID: 509160C19F5059FAA272B38986056F4DF3C0EB536516F4E14EA934EB708E1A32 Session-ID-ctx: Master-Key: PSK identity: None PSK identity hint: None SRP username: None Start Time: 1700654113 Timeout : 7200 (sec) Verify return code: 0 (ok) Extended master secret: no --- 

Compare that to connecting with s_client again but to tamu.edu and you’ll see that a pile of intermediate certs are sent along with the response for the parent site, which is correct.

For other neat libraries, katipo is a high-performance NIF to libcurl.

[dogweather]

Texas A&M fixed their configuration. I’m not used to large orgs like this taking outside reports seriously. Pretty nice.