HTTPS proxy with g_socket_client_connect_async

Hi all,
I am tying to debug usage of g_socket_client_connect_async on an address created with g_proxy_address_new , with an HTTPS proxy, which is failing with “Unacceptable TLS certificate”, despite the certificate tracking back to a system-trusted CA

The proxy works fine with chrome/firefox, and supplies a full certificate chain to a standard root CA (ISRG-Root-X1 via letsencrypt R3) (can be seen in chrome/firefox, although I cannot see this in wireshark)
I have also tried using htt ps://www.google.com:443 as the proxy, with the same results (of course it would have failed later even if TLS setup had been successful)

The problematic application I’m debugging is spice-gtk (spice / spice-gtk · GitLab)

  • Are HTTPS proxies supported at all?
  • Is there any way of getting more debug info?
  • Do we maybe need to somehow populate the ca store for the proxy?

Thanks a lot!

Error trace in spicy (just in case):
user@host $ SPICE_PROXY=https://www.google.com:443 G_MESSAGES_DEBUG=all SPICE_DEBUG=1 …/spicy-0.39.0.glibc2.14-x86_64.AppImage --host redacted --port 1234 --password redacted --spice-debug
(spicy:23220): GSpice-DEBUG: 22:32:27.283: …/…/src/spice-session.c:286 New session (compiled from package spice-gtk 0.39)
(spicy:23220): GSpice-DEBUG: 22:32:27.283: …/…/src/spice-session.c:290 Supported channels: main, display, inputs, cursor, playback, record, smartcard, usbredir, webdav
(spicy:23220): GSpice-DEBUG: 22:32:27.285: …/…/src/usb-device-manager.c:391 auto-connect filter set to 0x03,-1,-1,-1,0|-1,-1,-1,-1,1
(spicy:23220): GSpice-DEBUG: 22:32:27.285: …/…/src/usb-backend.c:437 spice_usb_backend_new >>
(spicy:23220): GSpice-DEBUG: 22:32:27.291: …/…/src/usb-backend.c:456 spice_usb_backend_new <<
[snip usb devices]
(spicy:23220): GSpice-DEBUG: 22:32:27.292: …/…/src/usb-backend.c:463 handle_libusb_events >>
(spicy:23220): GSpice-DEBUG: 22:32:27.292: …/…/tools/spicy.c:1881 connection_new (1)
(spicy:23220): GSpice-DEBUG: 22:32:27.292: …/…/src/spice-session.c:1833 no migration in progress
Spice-INFO: 22:32:27.321: …/…/src/channel-main.c:342:spice_main_set_property: SpiceMainChannel::color-depth has been deprecated. Property is ignored
(spicy:23220): GSpice-DEBUG: 22:32:27.321: …/…/src/spice-channel.c:140 main-1:0: spice_channel_constructed
(spicy:23220): GSpice-DEBUG: 22:32:27.321: …/…/src/spice-session.c:2328 main-1:0: new main channel, switching
(spicy:23220): GSpice-DEBUG: 22:32:27.321: …/…/src/spice-gtk-session.c:1565 Changing main channel from (nil) to 0x55e197db53e0
(spicy:23220): GSpice-DEBUG: 22:32:27.321: …/…/tools/spicy.c:1758 new channel (#0)
(spicy:23220): GSpice-DEBUG: 22:32:27.321: …/…/tools/spicy.c:1761 new main channel
[snip usb devices]
(spicy:23220): GSpice-DEBUG: 22:32:27.322: …/…/src/spice-channel.c:2710 main-1:0: Open coroutine starting 0x55e197db53e0
(spicy:23220): GSpice-DEBUG: 22:32:27.322: …/…/src/spice-channel.c:2540 main-1:0: Started background coroutine 0x55e197db51b0
(spicy:23220): GSpice-DEBUG: 22:32:27.322: …/…/src/spice-session.c:2265 main-1:0: Using plain text, port 1234
(spicy:23220): GSpice-DEBUG: 22:32:27.322: …/…/src/spice-session.c:2211 (with proxy https://www.google.com:443)
(spicy:23220): GSpice-DEBUG: 22:32:27.363: …/…/src/spice-session.c:2135 proxy lookup ready
(spicy:23220): GSpice-DEBUG: 22:32:27.363: …/…/src/spice-session.c:2118 main-1:0: connecting 0x7f347e26ddf0…
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-session.c:2102 main-1:0: connect ready
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-session.c:2277 main-1:0: open host: Unacceptable TLS certificate
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-channel.c:2570 main-1:0: Connect error
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-channel.c:2683 main-1:0: Coroutine exit main-1:0
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-channel.c:2873 main-1:0: reset
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/channel-main.c:1590 agent connected: no
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-channel.c:2821 main-1:0: channel reset
(spicy:23220): GSpice-DEBUG: 22:32:27.514: …/…/src/spice-channel.c:2428 main-1:0: Delayed unref channel 0x55e197db53e0
GSpice-Message: 22:32:27.514: main channel: failed to connect
GSpice-Message: 22:32:27.514: channel error: Unacceptable TLS certificate
[snip]

They are, but I doubt any developers have attempted to use them in years. You’re probably now our foremost expert on HTTPS proxies. Congratulations?

You can try running with G_MESSAGES_DEBUG=GLib-GIO in the environment and see if it prints debug from GSocketClient. But I’ve been trying that, and it doesn’t seem to work for some reason. Not sure why.

In my experience, debugging an issue like this generally requires adding printfs to the code until we find where exactly it is going wrong.

I doubt it. More likely, the code is expecting the proxy to present a certificate that’s valid for the destination address rather than for the proxy address. I think that would be a GLib bug. I’m just guessing though.

I found this explanation of how HTTPS proxies work. I assume you’re trying to use a “forwarding proxy” and not a “terminating proxy,” in the terminology of that blog post. It looks like GSocketClient is doing a TLS handshake with the endpoint, not with the HTTPS proxy. The proxy server is just in the middle. I guess speaking to the proxy might itself require HTTPS, though? Not sure if we support that or not, but if so, that would be a second layer of TLS. Either way, no special certificate configuration should be required.

But the blog post follows with an explanation of a “terminating proxy” which would definitely require some special root certificate configuration on your part, since that’s effectively a MITM attack and you would need the MITM certificate installed for it to work. Is that what you’re trying to do?

Hi Michael, thanks for your reply,

[…]You’re probably now our foremost expert on HTTPS proxies. Congratulations?

Um, Yay? :laughing:

You can try running with G_MESSAGES_DEBUG=GLib-GIO […]

Indeed, I’m not having any luck with that var either :frowning:

the blog post follows with an explanation of a “terminating proxy” […]

No, the usecase doesn’t fit their “terminating proxy” example: I’m connecting to the proxy with https (TLS), and asking for a TCP tunnel to the destination (“CONNECT host:port”) over that TLS connection; We aren’t doing any MITM in this case.

.

It seems like the spice-gtk developers actually tested “https proxies”, as it is mentioned in the commit-logs back in in 2014, and has a little specific handling in the code (setting the proper g_proxy_address_new params)
However I’m not sure what exactly their setup was.
g_socket_client_connect_async certainly seems to like it should support it, since it sends a TLS handshake when “protocol” in g_proxy_address_new is “https”

In the wild, https seems rarely used to explicitly connect with a proxy, rather only with the final endpoint; although with transparent proxies (incl MITM proxies that impersonate the endpoint), the proxy intercepts the TLS handshake that was meant for the endpoint (and may or may not impersonate it), but this is not the purpose of g_proxy_address_new, since transparent proxies require zero proxy-awareness in the clients (SSL MITM would require a system-trusted “evil” MITM CA of course)

Indeed, in your link, they are actually only using plain http proxies, in various configurations, and requesting that they access https websites with the CONNECT method; the SPICE scenario is somewhat different:

  • Establish a tcp connection to the proxy (same for both http & https proxies)
  • If https proxy, perform a TLS handshake with the proxy
  • Request a CONNECT to the endpoint
  • From now on, the proxy should pass all data through the connection unchanged (since it’s not doing any MITM magic): plain for http, or encrypted on the client side if we connected with TLS
  • This data could be anything, g_socket_client does not care if this is http, http+tls, or anything else (in my case the SPICE protocol)
    (If we were using a https connection to the proxy to access an https website in this setup, the data would end up being doubly-encrypted on the wire between the client and the proxy, but this is not handled by g_socket_client_connect_async alone)

Indeed, wireshark show that this is what is happening:

  • for a http proxy, send a CONNECT final_host:final_port
  • for https, attempt a TLS handshake with the proxy (and we never get any further)

Since g_socket_client does not care what the underlying data will be (http, http+tls, whatever else), it shouldn’t be trying to authenticate the endpoint, but that might indeed be the bug.

While digging, I noticed that g_proxy_address_new has no way of knowing the hostname of the proxy server, only its IP, which means that my domain-only certificate cannot be valid anyways… (letsencrypt does not provide certs for IP addresses, only domain names)
I have tried using a system-trusted certificate that DOES authenticate the proxy’s IP (signed by my own CA, that is installed in the system CAs) , but that also fails in the same manner, so maybe that’s not exactly the (only?..) problem…

I have tried using the same hostname as the proxy for the final destination endpoint (g_proxy_address_new “dest_hostname”), in case that is being erroneously used as the expected identity of the proxy, but to no avail.
g_socket_client_connect_async Never sends any SNI info (TLS server_name extension), which could either mean it does not know what hostname to expect from the proxy, or simply that SNI just isn’t supported.

It certainly looks like delving into the source of g_socket_client_connect_async and printf()s is going to be the only way out of this…

For the moment I have given up, and am handling the TLS part with a helper program: it sets up a TCP server on localhost, that pipes any incoming connections into a new TLS connection to the proxy; spicy is then instructed to use this local server as a normal http proxy.
Works like a charm, but it’s a pity. Maybe curiosity pull be back down the rabbit hole :wink:

BR, Jonathan

My worry is that GSocketClient works in four steps: resolving, connecting, proxy negotiating, and TLS handshaking, in that order. And it sounds like for this to work, the TLS handshaking would need to come first.

Hm, that means you don’t want GLib to perform any certificate verification at all on the connection to the endpoint server? Hmmmm, that seems… dubious.

But this, at least, is simple and should not be fail regardless of anything else. I wonder what we’re doing wrong here.

Honestly I’m not sure if that makes sense or not for TLS connections. Because if GSocketClient doesn’t authenticate the endpoint, then you have no way to do the authentication at all, and that is surely not OK. Right? But anyway, I understand this is not the step that’s failing for you. You’re seeing the failure earlier when authenticating the proxy server itself, yes?

Ouch. Well, likely as not that explains it…

SNI is absolutely supported. We would be getting a lot of complaints if that were to break.

Yes, though I would start with g_socket_client_connect() instead because the code is significantly simpler. I’d be willing to investigate a little to see if anything obvious is wrong if you know of a public server that I should try testing with.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.