Proxy throwing PrematureChannelClosureException with basic settings

[funrep]

Describe the bug
When using a proxy with 3.0.0-RC6 I get the following error: io.netty.handler.codec.PrematureChannelClosureException: Channel closed while executing the request. This is likely caused due to a client connection misconfiguration. I'm running this on macOS 14.4.1.

To Reproduce
Start local instance of Squid:

docker run --platform linux/amd64 -d -p 3128:3128 wernight/squid

Verify with cURL (--proxytunnel to force CONNECT) that proxy works

curl --proxytunnel -x http://localhost:3128 -X 'GET' https://example.com
<gets back html of example.com>
docker exec -i -t <container_id> cat /var/log/squid/access.log
<gets back log>

Code

import zio._

import zio.http._
import zio.http.netty.NettyConfig
import zio.http.netty.client.NettyClientDriver

object HttpsClient extends ZIOAppDefault {
  val url = URL.decode("https://example.com/").toOption.get

  val sslConfig = ClientSSLConfig.Default
  val clientConfig = ZClient.Config.default.ssl(sslConfig).proxy(Proxy(URL.decode("http://localhost:3128").right.get))

  val program = for {
    res  <- ZClient.request(Request.get(url))
    data <- res.body.asString
    _    <- Console.printLine(data)
  } yield ()

  val run =
    program.provide(
      ZLayer.succeed(clientConfig),
      Client.customized,
      NettyClientDriver.live,
      DnsResolver.default,
      ZLayer.succeed(NettyConfig.default),
      Scope.default,
    )
}

build.sbt

scalaVersion := "3.3.1"
libraryDependencies += "dev.zio" %% "zio" % "2.1.0-RC3"
libraryDependencies += "dev.zio" %% "zio-http" % "3.0.0-RC6"
libraryDependencies += "io.netty" % "netty-codec-http"              % "4.1.109.Final"
libraryDependencies += "io.netty" % "netty-handler-proxy"           % "4.1.109.Final"
libraryDependencies += "io.netty" % "netty-transport-native-epoll"  % "4.1.109.Final"
libraryDependencies += "io.netty" % "netty-transport-native-kqueue" % "4.1.109.Final"
javaOptions ++= Seq(
  "-Djava.net.preferIPv4Stack=true"
)

squid.conf

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8	# RFC1918 possible internal network
acl localnet src 172.16.0.0/12	# RFC1918 possible internal network
acl localnet src 192.168.0.0/16	# RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
#http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
#http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
#http_access deny all

# Squid normally listens to port 3128
http_port 3128

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320

docker exec -i -t <container_id> cat /var/log/squid/access.log | wc
0
sbt run
<error put below>
docker exec -i -t <container_id> cat /var/log/squid/access.log
<redacted>     93 <redacted> TCP_TUNNEL/200 0 CONNECT example.com:443 - HIER_DIRECT/<redacted> -

Error and stack trace:

timestamp=2024-05-03T06:34:15.539852Z level=ERROR thread=#zio-fiber-1622405021 message="" cause="Exception in thread "zio-fiber-1424227775" io.netty.handler.codec.PrematureChannelClosureException: Channel closed while executing the request. This is likely caused due to a client connection misconfiguration
        at zio.http.netty.client.NettyClientDriver.requestOnChannel$$anonfun$1$$anonfun$2$$anonfun$2$$anonfun$1(NettyClientDriver.scala:171)
        at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$1(ZIO.scala:5196)
        at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$adapted$1(ZIO.scala:5196)
        at <empty>.HttpsClient.program(Main.scala:15)
        at <empty>.HttpsClient.program(Main.scala:18)
        at <empty>.HttpsClient.run(Main.scala:21)"
[error] Exception in thread "zio-fiber-1424227775" io.netty.handler.codec.PrematureChannelClosureException: Channel closed while executing the request. This is likely caused due to a client connection misconfiguration
[error]         at zio.http.netty.client.NettyClientDriver.requestOnChannel$$anonfun$1$$anonfun$2$$anonfun$2$$anonfun$1(NettyClientDriver.scala:171)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$1(ZIO.scala:5196)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$adapted$1(ZIO.scala:5196)
[error]         at <empty>.HttpsClient.program(Main.scala:15)
[error]         at <empty>.HttpsClient.program(Main.scala:18)
[error]         at <empty>.HttpsClient.run(Main.scala:21)
[error]         at zio.http.netty.client.NettyClientDriver.requestOnChannel$$anonfun$1$$anonfun$2$$anonfun$2$$anonfun$1(NettyClientDriver.scala:171)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$1(ZIO.scala:5196)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$adapted$1(ZIO.scala:5196)
[error]         at <empty>.HttpsClient.program(Main.scala:15)
[error]         at <empty>.HttpsClient.program(Main.scala:18)
[error]         at <empty>.HttpsClient.run(Main.scala:21)
[error] Caused by: io.netty.handler.codec.PrematureChannelClosureException: Channel closed while executing the request. This is likely caused due to a client connection misconfiguration
[error]         at zio.http.netty.client.NettyClientDriver.requestOnChannel$$anonfun$1$$anonfun$2$$anonfun$2$$anonfun$1(NettyClientDriver.scala:171)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$1(ZIO.scala:5196)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$adapted$1(ZIO.scala:5196)
[error]         at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:949)
[error]         at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:380)
[error]         at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:452)
[error]         at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:214)
[error]         at zio.internal.FiberRuntime.run(FiberRuntime.scala:141)
[error]         at zio.internal.ZScheduler$$anon$3.run(ZScheduler.scala:437)
[error] stack trace is suppressed; run last Compile / run for the full output
[error] (Compile / run) Exception in thread "zio-fiber-1424227775" io.netty.handler.codec.PrematureChannelClosureException: Channel closed while executing the request. This is likely caused due to a client connection misconfiguration
[error]         at zio.http.netty.client.NettyClientDriver.requestOnChannel$$anonfun$1$$anonfun$2$$anonfun$2$$anonfun$1(NettyClientDriver.scala:171)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$1(ZIO.scala:5196)
[error]         at zio.ZIO$ZIOBooleanOps$.$amp$amp$extension$$anonfun$adapted$1(ZIO.scala:5196)
[error]         at <empty>.HttpsClient.program(Main.scala:15)
[error]         at <empty>.HttpsClient.program(Main.scala:18)
[error]         at <empty>.HttpsClient.run(Main.scala:21)
[error] Total time: 4 s, completed 3 May 2024, 08:34:15

Expected behaviour
Get back response from https://example.com

Desktop (please complete the following information):

• OS: macOS 14.4.1
• Version: 3.0.0-RC6
• Squid 3.5.23
• Scala 3.3.1

Would be good if you could take a look. It's currently blocking roll out of zio-http for us.

EDIT: Added scala version

[987Nabil]

See #2841

[funrep]

@987Nabil I built zio-http from main branch with the references changes and the same issues described above persists.

[kyri-petrou]

@987Nabil I did a small investigation on this; I highly doubt this is related to ZIO at all, but rather something with how we're creating the netty pipeline. For some reason that I can't figure out, the HttpClientCodec handler (which comes from Netty) is closing the Netty channel 🫤