[Bug]: OIDC Discovery for OAuth2 Proxy not working

[testgoofy]

Preflight Checklist

• I could not find a solution in the documentation, the existing issues or discussions
• I have joined the ZITADEL chat

Environment

Self-hosted

Version

v2.51.3

Database

PostgreSQL

Database Version

16.3

Describe the problem caused by this bug

I have a OAuth2 Proxy in front of another web application.
The OAuth2 Proxy links to the Zitadel instance.
When deployed the OAuth2 Proxy crashes with the error

[2024/05/16 18:06:01] [main.go:60] ERROR: Failed to initialise OAuth2 Proxy: error intiailising provider: could not create provider data: error building OIDC ProviderVerifier: could not get verifier builder: error while discovery OIDC configuration: failed to discover OIDC configuration: error performing request: Get "http://localhost:8080/.well-known/openid-configuration": dial tcp [::1]:8080: connect: connection refused

When testing the same setup with another identity provider, it works
The issues also persists, when I use a Zitadel Cloud account.

To reproduce

1. Create a file docker-compose.yaml with the content:

services:

  auth:
    image: quay.io/oauth2-proxy/oauth2-proxy:v7.3.0
    restart: unless-stopped
    command: --http-address 0.0.0.0:4180
    environment:
      - OAUTH2_PROXY_UPSTREAMS=http://app:80/
      - OAUTH2_PROXY_PROVIDER_DISPLAY_NAME=Zitadel
      - OAUTH2_PROXY_PROVIDER=oidc
      - OAUTH2_PROXY_OIDC_ISSUER_URL=http://localhost:8080
      - OAUTH2_PROXY_CLIENT_ID=267404730723008514@development_&_testing
      - OAUTH2_PROXY_CLIENT_SECRET=qY3JnpdgfW6sKPxbKx0fHoPxqbhzDhuXbujI174c33hR6LMoDg4KPlwnwqYxAGC0
      - OAUTH2_PROXY_EMAIL_DOMAINS=*
      - OAUTH2_PROXY_REDIRECT_URL=http://localhost:4180/oauth2/callback
      - OAUTH2_PROXY_COOKIE_SECRET=Ak5Q5kx-Ukg11yxnoPxV1cqi3QtlMhDGZp722GCb840=
      - OAUTH2_PROXY_SHOW_DEBUG_ON_ERROR=true
      - OAUTH2_PROXY_COOKIE_SECURE=false
      # - OAUTH2_PROXY_REDIS_CONNECTION_URL=redis://memory-cache
      # - OAUTH2_PROXY_SESSION_STORE_TYPE=redis
    depends_on:
      - idenitty-provider
      # - memory-cache
    ports:
      - 4180:4180

  # memory-cache:
  #   image: redis:6.2.7
  #   restart: always

  app:
    image: traefik/whoami
    restart: unless-stopped

  idenitty-provider:
    image: ghcr.io/zitadel/zitadel:v2.51.3 
    restart: unless-stopped
    command: start-from-init --masterkeyFromEnv
    environment:
      - 'ZITADEL_LOGSTORE_ACCESS_STDOUT_ENABLED=true'
      - 'ZITADEL_DATABASE_POSTGRES_HOST=database'
      - 'ZITADEL_DATABASE_POSTGRES_PORT=5432'
      - 'ZITADEL_DATABASE_POSTGRES_DATABASE=db_zitadel'
      - 'ZITADEL_DATABASE_POSTGRES_USER_USERNAME=dbu_zitadel'
      - 'ZITADEL_DATABASE_POSTGRES_USER_PASSWORD=DB_USER_PASSWORD'
      - 'ZITADEL_DATABASE_POSTGRES_USER_SSL_MODE=disable'
      - 'ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME=root'
      - 'ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD=DB_ADMIN_PASSWORD'
      - 'ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE=disable'
      - 'ZITADEL_EXTERNALDOMAIN=localhost'
      - 'ZITADEL_EXTERNALPORT=8080'
      - 'ZITADEL_EXTERNALSECURE=false'
      - 'ZITADEL_MASTERKEY=MasterkeyNeedsToHave32Characters'
      - 'ZITADEL_TLS_ENABLED=false'
    depends_on:
      database:
        condition: 'service_healthy'
    ports:
      - 8080:8080

  database:
    image: postgres:16.3
    restart: unless-stopped
    environment:
      - POSTGRES_USER=root
      - POSTGRES_PASSWORD=DB_ADMIN_PASSWORD
      - POSTGRES_DB=db_zitadel
    healthcheck:
      test: ["CMD-SHELL", "pg_isready", "-d", "db_zitadel", "-U", "root"]
      interval: '10s'
      timeout: '30s'
      retries: 5
      start_period: '20s'

2. Start the stack with docker compose up -d
3. OAuth2 Proxy crashes

Screenshots

No response

Expected behavior

As written in the Documentation: OAuth2 Proxy starts flawless

Operating System

No response

Relevant Configuration

See docker-compose.yaml

Additional Context

I tried to replace OAUTH2_PROXY_OIDC_ISSUER_URL=http://localhost:8080 with the hardcoded endpoints

- OAUTH2_PROXY_OIDC_ISSUER_URL=NOT_USED
- OAUTH2_PROXY_LOGIN_URL=http://localhost:8080/oauth/v2/authorize
- OAUTH2_PROXY_REDEEM_URL=http://localhost:8080/oauth/v2/token
- OAUTH2_PROXY_OIDC_JWKS_URL=http://localhost:8080/oauth/v2/keys
- OAUTH2_PROXY_SKIP_OIDC_DISCOVERY=true

This gives me another error when logging in.
Same error as in #5267.