IPv6 proxy weirdness #4180

dharapvj · 2024-04-24T18:58:39Z

Bug Description

If frpc connects to frps server via IPv6 - I cannot connect to local services exposed via container e.g. nginx docker container. But same frpc connects to frps via IPv4, same local service can get connected.

More details can be found in the reproduction steps.

frpc Version

0.57.0

frps Version

0.57.0

System Architecture

linux/amd (client) linux/arm64 (server)

Configurations

########  frps.toml - server_ipv6
bindAddr = "::"
bindPort = 2333
log.level = "debug"

########  frps.ini - server_ipv4
[common]
bind_port = 2333

######### frpc.toml
serverAddr = "IPv6_IP_ADDR_OF_SERVER_IPv6"
serverPort = 2333
log.level = "trace"

[[proxies]]
name = "ssh"
type = "tcp"
localIp = "127.0.0.1"
localPort = 22
remotePort = 43332

[[proxies]]
name = "web"
type = "tcp"
localIp = "127.0.0.1"
localPort = 9000

Logs

Logs:

❯ frpc -c /etc/frp/frpc.toml
2024-04-25 00:10:33.130 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.toml]
2024-04-25 00:10:33.130 [I] [client/service.go:294] try to connect to server...
2024-04-25 00:10:33.154 [I] [client/service.go:286] [3dede59ef6706e74] login to server success, get run id [3dede59ef6706e74]
2024-04-25 00:10:33.154 [I] [proxy/proxy_manager.go:173] [3dede59ef6706e74] proxy added: [ssh web]
2024-04-25 00:10:33.154 [T] [proxy/proxy_wrapper.go:200] [3dede59ef6706e74] [ssh] change status from [new] to [wait start]
2024-04-25 00:10:33.154 [T] [proxy/proxy_wrapper.go:200] [3dede59ef6706e74] [web] change status from [new] to [wait start]
2024-04-25 00:10:33.154 [D] [client/control.go:244] [3dede59ef6706e74] send heartbeat to server
2024-04-25 00:10:33.160 [I] [client/control.go:170] [3dede59ef6706e74] [ssh] start proxy success
2024-04-25 00:10:33.165 [W] [client/control.go:168] [3dede59ef6706e74] [web] start error: port unavailable
2024-04-25 00:10:33.165 [D] [client/control.go:195] [3dede59ef6706e74] receive heartbeat from server
2024-04-25 00:11:03.155 [D] [client/control.go:244] [3dede59ef6706e74] send heartbeat to server
2024-04-25 00:11:03.160 [D] [client/control.go:195] [3dede59ef6706e74] receive heartbeat from server
2024-04-25 00:11:03.172 [T] [proxy/proxy_wrapper.go:200] [3dede59ef6706e74] [web] change status from [start error] to [wait start]
2024-04-25 00:11:03.177 [W] [client/control.go:168] [3dede59ef6706e74] [web] start error: port unavailable

Logs when connected to IPv4 Server:

frpc[192002]: WARNING: ini format is deprecated and the support will be removed in the future, please use yaml/json/toml format instead!
frpc[192002]: 2024-04-25 00:19:30.606 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.ini]
frpc[192002]: 2024-04-25 00:19:30.607 [I] [client/service.go:294] try to connect to server...
frpc[192002]: 2024-04-25 00:19:30.633 [I] [client/service.go:286] [055be5915b8650e0] login to server success, get run id [055be5915b8650e0]
frpc[192002]: 2024-04-25 00:19:30.634 [I] [proxy/proxy_manager.go:173] [055be5915b8650e0] proxy added: [ssh tcp_port_0 tcp_port_1]
frpc[192002]: 2024-04-25 00:19:30.634 [T] [proxy/proxy_wrapper.go:200] [055be5915b8650e0] [ssh] change status from [new] to [wait start]
frpc[192002]: 2024-04-25 00:19:30.634 [T] [proxy/proxy_wrapper.go:200] [055be5915b8650e0] [tcp_port_1] change status from [new] to [wait start]
frpc[192002]: 2024-04-25 00:19:30.634 [T] [proxy/proxy_wrapper.go:200] [055be5915b8650e0] [tcp_port_0] change status from [new] to [wait start]
frpc[192002]: 2024-04-25 00:19:30.635 [D] [client/control.go:244] [055be5915b8650e0] send heartbeat to server
frpc[192002]: 2024-04-25 00:19:30.642 [I] [client/control.go:170] [055be5915b8650e0] [ssh] start proxy success
frpc[192002]: 2024-04-25 00:19:30.642 [I] [client/control.go:170] [055be5915b8650e0] [tcp_port_1] start proxy success
frpc[192002]: 2024-04-25 00:19:30.642 [I] [client/control.go:170] [055be5915b8650e0] [tcp_port_0] start proxy success
frpc[192002]: 2024-04-25 00:19:30.646 [D] [client/control.go:195] [055be5915b8650e0] receive heartbeat from server
frpc[192002]: 2024-04-25 00:19:47.380 [D] [proxy/proxy_wrapper.go:260] [055be5915b8650e0] [tcp_port_1] start a new work connection, localAddr: 192.168.1.52:34486 remoteAddr: 13.235.216.249:2333

Steps to reproduce

I have frps running on a IPv6 enabled server (server_ipv6) with minimal config as below in AWS
I have another frps running on another IPv4 enabled server (server_ipv4) with same config in AWS
I have frpc running in my local machine.
I am trying to proxy a web-server running in my local running inside a container with ports published to host. e.g. podman run -p 9000:9000 -p 9001:9001 quay.io/minio/minio server /data --console-address ":9001" But anything else like nginx will also do. I can access service via curl -v localhost:9000 on the local machine host.
If I connect my frpc to IPv4 server's frps, everything work fine.
If I connect my frpc to IPv6 server's frps, I get port unavailable error.

Affected area

The text was updated successfully, but these errors were encountered:

fatedier · 2024-04-25T12:19:13Z

2024-04-25 00:11:03.177 [W] [client/control.go:168] [3dede59ef6706e74] [web] start error: port unavailable

You are not using the same configuration?

You need to check the error messages in the frps log.

dharapvj · 2024-04-25T17:33:34Z

I think it was late in night and I pasted wrong logs.. I recreated the scenario again. I also removed ssh proxy so that logs can be simple to look at.

Here is config:
ipv4 server

[common]
bind_port = 2333

ipv6 server

bindAddr = "::"
bindPort = 2333
log.level = "debug"

local client config

serverAddr = "<IPv6_address>"
# serverAddr = "<IPv4_Address>"
serverPort = 2333
log.level = "trace"

[[proxies]]
name = "web"
type = "tcp"
localIp = "127.0.0.1"
localPort = 80
remotePort = 80

Here are logs:
Connecting to ipv4 machine:

❯ frpc -c /etc/frp/frpc.toml
2024-04-25 22:37:43.498 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.toml]
2024-04-25 22:37:43.498 [I] [client/service.go:294] try to connect to server...
2024-04-25 22:37:43.521 [I] [client/service.go:286] [a280a2e63b59f284] login to server success, get run id [a280a2e63b59f284]
2024-04-25 22:37:43.521 [I] [proxy/proxy_manager.go:173] [a280a2e63b59f284] proxy added: [web]
2024-04-25 22:37:43.521 [T] [proxy/proxy_wrapper.go:200] [a280a2e63b59f284] [web] change status from [new] to [wait start]
2024-04-25 22:37:43.521 [D] [client/control.go:244] [a280a2e63b59f284] send heartbeat to server
2024-04-25 22:37:43.527 [I] [client/control.go:170] [a280a2e63b59f284] [web] start proxy success
2024-04-25 22:37:43.527 [D] [client/control.go:195] [a280a2e63b59f284] receive heartbeat from server
^C

Connecting to ipv6 machine

❯ frpc -c /etc/frp/frpc.toml
2024-04-25 23:00:17.651 [I] [sub/root.go:142] start frpc service for config file [/etc/frp/frpc.toml]
2024-04-25 23:00:17.651 [I] [client/service.go:294] try to connect to server...
2024-04-25 23:00:17.684 [I] [client/service.go:286] [791c87b2fe506ae9] login to server success, get run id [791c87b2fe506ae9]
2024-04-25 23:00:17.684 [I] [proxy/proxy_manager.go:173] [791c87b2fe506ae9] proxy added: [web]
2024-04-25 23:00:17.684 [T] [proxy/proxy_wrapper.go:200] [791c87b2fe506ae9] [web] change status from [new] to [wait start]
2024-04-25 23:00:17.685 [D] [client/control.go:244] [791c87b2fe506ae9] send heartbeat to server
2024-04-25 23:00:17.690 [W] [client/control.go:168] [791c87b2fe506ae9] [web] start error: port unavailable
2024-04-25 23:00:17.690 [D] [client/control.go:195] [791c87b2fe506ae9] receive heartbeat from server
^C

Corresponding ipv6 server logs:

2024-04-25 17:30:13.847 [I] [frps/root.go:105] frps uses config file: frps.toml
2024-04-25 17:30:14.872 [I] [server/service.go:237] frps tcp listen on [::]:2333
2024-04-25 17:30:14.872 [I] [frps/root.go:114] frps started successfully
2024-04-25 17:30:17.680 [I] [server/service.go:575] [791c87b2fe506ae9] client login info: ip [[2402:b00:e:f009:1059:f37a:5f71:155f]:60050] version [0.57.0] hostname [] os [linux] arch [amd64]
2024-04-25 17:30:17.686 [W] [server/control.go:396] [791c87b2fe506ae9] new proxy [web] type [tcp] error: port unavailable
2024-04-25 17:30:17.687 [D] [server/control.go:432] [791c87b2fe506ae9] receive heartbeat
2024-04-25 17:30:17.687 [D] [server/control.go:243] [791c87b2fe506ae9] new work connection registered
2024-04-25 17:30:21.480 [D] [server/service.go:524] Accept new mux stream error: EOF
2024-04-25 17:30:21.480 [I] [server/control.go:359] [791c87b2fe506ae9] client exit success

The service is available locally

❯ curl -I 127.0.0.1
HTTP/1.1 200 OK
Server: openresty
Date: Thu, 25 Apr 2024 17:10:36 GMT
Content-Type: text/html
Content-Length: 1033
Last-Modified: Wed, 28 Feb 2024 22:25:56 GMT
Connection: keep-alive
ETag: "65dfb2f4-409"
Accept-Ranges: bytes

xqzr · 2024-04-25T19:27:12Z

看起来 frps 所在的服务器，80 端口被其他进程占用
使用 ss -tlnp sport 80 找到占用 80 端口的进程

xqzr · 2024-04-25T19:28:14Z

另外

bindAddr = "::"

不是必须的

fatedier · 2024-04-26T03:04:49Z

@dharapvj You can refer to the comments above to confirm if port 80 has been occupied and try removing bindAddr.

dharapvj · 2024-04-28T17:31:28Z

port 80 is of course occupied on the client (frpc) machine. That's the port on which I want incoming traffic from the frps machine to get routed at. But please note the fact that if I change the server address in frpc config to an IPv4 adddress instead of an IPv6 address and everything works fine. Then frpc does not complain about port not being available!
if I remove bindAddr - it tries to by default connect on IPv4 0.0.0.0. But my target server machine is in IPv6-only network where IPv4 traffic cannot reach.

I just request you to try out such a scenario once. I have given detailed information and logs so that it could be easier for you to try and reproduce. Can you please try reproducing the issue once?

fatedier · 2024-04-29T03:57:17Z

You cannot expect everyone to have the same environment as you do, there are tests related to IPv6 in end-to-end testing, and obvious errors will be directly detected.

If possible, please conduct more tests, try more ports or other configuration adjustments. Otherwise, I won't be able to provide much help.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IPv6 proxy weirdness #4180

IPv6 proxy weirdness #4180

dharapvj commented Apr 24, 2024

fatedier commented Apr 25, 2024

dharapvj commented Apr 25, 2024

xqzr commented Apr 25, 2024

xqzr commented Apr 25, 2024

fatedier commented Apr 26, 2024

dharapvj commented Apr 28, 2024

fatedier commented Apr 29, 2024

IPv6 proxy weirdness #4180

IPv6 proxy weirdness #4180

Comments

dharapvj commented Apr 24, 2024

Bug Description

frpc Version

frps Version

System Architecture

Configurations

Logs

Steps to reproduce

Affected area

fatedier commented Apr 25, 2024

dharapvj commented Apr 25, 2024

xqzr commented Apr 25, 2024

xqzr commented Apr 25, 2024

fatedier commented Apr 26, 2024

dharapvj commented Apr 28, 2024

fatedier commented Apr 29, 2024