Skip to content

enhancement/bump-resources-on-dev #86

@bwalsh

Description

@bwalsh

Based on my search through the helm/revproxy folder, here are the configurations that could produce a 502 Bad Gateway error:

Timeout Configurations

Both nginx.conf and nginxPrivate/nginx.conf have identical timeout settings that could lead to 502 errors:

proxy_buffer_size          16k;
proxy_buffers              8 16k;
proxy_busy_buffers_size    32k;
client_body_buffer_size    16k;
proxy_read_timeout         400;
proxy_send_timeout         400;
proxy_connect_timeout      400;

Key timeout values:

  • proxy_read_timeout: 400 seconds - If backend doesn't respond within 400s, returns 502
  • proxy_send_timeout: 400 seconds - If backend doesn't accept data within 400s, returns 502
  • proxy_connect_timeout: 400 seconds - If connection to backend fails within 400s, returns 502
  • keepalive_timeout: 65 seconds (line 36)

Service-Specific Extended Timeouts

Some services override the default 400s with longer timeouts (600s):

Guppy Service

proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
send_timeout 600s;

Sheepdog Service (API)

proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;

Peregrine Service (GraphQL)

proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;

Workspace Services (Ambassador/Hatchery)

# for fixing noVNC connection timeout issue
proxy_read_timeout 36000s;

Rate Limiting

I didn't find explicit rate limiting directives (like limit_req_zone or limit_conn_zone) in the revproxy configuration, which could also cause 502-like errors when backends are overwhelmed.

Other 502 Triggers

  1. Buffer size limits - If headers/responses exceed buffer sizes:

    • proxy_buffer_size: 16k
    • large_client_header_buffers: 4 64k
    • client_header_buffer_size: 4k
  2. Backend service unavailability - When upstream services are down or unreachable

  3. Error page handling:

error_page 500 501 502 503 504 @5xx;

location @5xx {
    internal;
    return 500 "{ \"error\": \"service failure - try again later\"}";
}

Note: These results may be incomplete. For a comprehensive search, you can view more results in the GitHub code search UI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions