Built a Grafana dashboard with a $service variable. Selecting “All” broke the panel. Manually selecting every service one by one worked.

Turns out when the user picks “All,” Grafana sends the literal string All unless you configure otherwise. Our PromQL query was doing:

sum(rate(http_requests_total{service="$service"}[5m]))

Which became:

sum(rate(http_requests_total{service="All"}[5m]))

Which matches nothing.

Fixes:

  1. Custom all value in the variable config. Set it to .* for Prometheus:

    Custom all value: .*

    And use =~ in the query:

    http_requests_total{service=~"$service"}

    Now when “All” is selected, $service becomes .* which matches everything.

  2. Multi-select with regex. Set the variable as multi-value + regex. Then $service becomes (a|b|c) when multiple are selected. Pairs with =~ label matcher.

  3. Always use =~, never =, when using template variables. Even for single values. Regex-matching a fixed string is free; it covers the “All” case for free.

Pattern I now use for every multi-value template variable:

Include All option: yes
Custom all value: .*
Multi-value: yes
Label matcher: =~

And every panel’s query uses {label=~"$var"}. Works for All. Works for multi-select. Works for single select. Done.