Table of contents

An engineer used to working with regular servers comes to Kubernetes and does what they've always done: exports HTTP_PROXY into the environment, restarts the process, and waits for all egress traffic to flow through the proxy. Sometimes it works. Far more often it breaks half the cluster, while the other half keeps going straight to the internet, bypassing the proxy. And that's when things get interesting: why do some pods pick up the variables while others don't, why do services suddenly stop seeing each other, and why did the healthcheck suddenly go through a proxy server that isn't on the trusted list.

This article is a detailed breakdown of how egress traffic in a cluster actually works and where a proxy can be correctly inserted. We won't rehash the basics of setting environment variables: you already know that. The conversation will focus on the specifics of Kubernetes, where familiar approaches produce unexpected effects. All examples are based on real manifest fragments, and the proxy infrastructure of choice is Proxeon (proxeon.net).

The Basics: Why Things Are Different in a Cluster

Let's start with the fundamentals. On a classic server, the concept of egress traffic is trivial: there's one network interface, a routing table, system environment variables, and almost everything you run inherits them from the parent shell. Export a variable in the profile, and the entire user process sees it.

In Kubernetes, there's no single point like that. Here you have a pod — the smallest unit of deployment, inside which one or more containers live. The pod has its own network namespace, its own IP address, its own set of environment variables defined by the manifest. There's no shell from which a process could inherit something. An environment variable only appears in a container if you explicitly declared it in the pod spec or in the image.

What Is Pod Egress Traffic

When a container reaches out to an external address, the packet takes a long journey. First, it leaves the container's network namespace through a virtual interface. Then it lands in the node's network stack, where the CNI plugin — the component responsible for cluster networking — takes over. Next, iptables or eBPF rules come into play, along with the SNAT mechanism (source address translation), after which the packet exits through the node's network interface into the outside world.

The key point: from the external service's perspective, the request doesn't come from the pod's address but from the cluster node's address. The pod is hidden behind address translation. This is the first thing that surprises newcomers when they try to set up IP allowlisting: they add the pod's address to the list, but the traffic arrives from a completely different one.

Two Types of Egress Traffic

It's important to distinguish from the very start between two fundamentally different flows:

  • East-west — traffic between services inside the cluster. One pod talks to another through a service, ClusterIP, or a DNS name like my-service.namespace.svc.cluster.local.
  • North-south — traffic going outward, to external APIs, databases, partner services, object storage.

A proxy is almost always needed only for north-south traffic. And the most common and painful mistake is that a misconfigured proxy accidentally captures east-west traffic too, tearing apart internal communication. That's exactly why the exclusion list matters here more than anywhere else. But more on that later.

Deep Dive: Three Layers Where a Proxy Can Live

There are exactly three architectural layers where you can embed a proxy in a pod's egress path. Each solves the problem differently, each has its own cost. Let's go through all three honestly, with pros and cons.

Layer 1: The Container Itself

This is when the application inside the container knows about the proxy itself. It reads the HTTP_PROXY and HTTPS_PROXY environment variables or has a proxy in its own config and routes its HTTP requests through it. The proxy logic is baked into the application's own client library.

Pros. Maximum simplicity to get started. Nothing needs to be added to the cluster, no additional components. Control at the individual pod level: you know exactly which application talks to where.

Cons. Every application must be able to read these variables, and not all of them can. Configuration gets smeared across dozens of manifests. Updating the proxy address means going through every deployment. It's easy to forget one service, and it'll go direct. There's no centralized policy.

Layer 2: The Sidecar Container

Here, a second container — a sidecar — runs alongside the main one in the same pod. It intercepts egress traffic and routes it through the proxy. The application doesn't even need to know the proxy exists: it sends requests as usual, and the sidecar quietly proxies them. This is how service meshes like Istio and Linkerd work, and you can apply the same principle to run a lightweight local proxy agent.

Pros. Transparency for the application. A unified policy through the pod template. The ability to add observability on top of proxying: metrics, tracing, retries, timeouts. The sidecar is isolated in the pod and shares its lifecycle.

Cons. Overhead: now there are two containers per pod, meaning more memory and CPU. Debugging gets harder — there's an extra link in the chain. The order in which containers start matters: if the application starts before the sidecar, the first requests may fail. In Kubernetes 1.28+, this problem is solved by native sidecar containers as init containers with a restartPolicy of Always.

Layer 3: The Cluster Egress Gateway

This is a dedicated node or pod through which all cluster egress traffic is forcibly routed. Packets are routed to the egress gateway by the CNI, and the gateway either talks to an external proxy or acts as the exit point with a stable egress IP.

Pros. A single point of control for the entire cluster. A stable, predictable egress address that's easy to add to external partners' allowlists. Change the policy in one place. Applications know nothing.

Cons. The egress gateway becomes a critical point: if it goes down, all north-south traffic stops. You need redundancy and monitoring. Setup is more complex and requires CNI support. Granularity is lower: it's harder to set different policies for different applications without extra rules.

How to Choose a Layer

A practical rule of thumb from experience: a small project with a couple of services that need an external proxy — container layer. A medium cluster with dozens of services and a requirement for a unified policy — sidecar. A large infrastructure where external partners demand a fixed egress IP and full traffic auditing — egress gateway. Often layers are combined: an egress gateway for baseline control plus environment variables for fine-tuning individual pods through Proxeon.

Environment Variables in the Manifest and the Role of NO_PROXY

Let's move to the most underappreciated detail. The HTTP_PROXY, HTTPS_PROXY, and NO_PROXY variables in a manifest are set through the container spec's env block. Classic names are usually duplicated in lowercase, because some libraries only read the lowercase versions.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: worker
spec:
 replicas: 2
 selector:
matchLabels:
 app: worker
 template:
metadata:
 labels:
app: worker
spec:
 containers:
- name: app
 image: registry.example.com/worker:1.4.0
 env:
- name: HTTP_PROXY
 value: "http://gate.proxeon.net:8080"
- name: HTTPS_PROXY
 value: "http://gate.proxeon.net:8080"
- name: NO_PROXY
 value: "localhost,127.0.0.1,10.0.0.0/8,.svc,.svc.cluster.local,.cluster.local,kubernetes.default"
- name: http_proxy
 value: "http://gate.proxeon.net:8080"
- name: https_proxy
 value: "http://gate.proxeon.net:8080"
- name: no_proxy
 value: "localhost,127.0.0.1,10.0.0.0/8,.svc,.svc.cluster.local,.cluster.local,kubernetes.default"

Why the Cluster Falls Apart Without a Proper NO_PROXY

Here's the crux of the problem. As soon as you declare HTTP_PROXY, the application's HTTP client starts routing absolutely all requests through the proxy, including calls to neighboring services inside the cluster. But internal services are only reachable within the cluster network — the external proxy server can't physically reach them. Result: a request to orders.default.svc.cluster.local goes to the external proxy, which tries to resolve this name, can't, and returns an error. Internal communication breaks instantly.

NO_PROXY is the exclusion list — addresses and domains that should bypass the proxy and go direct. On a regular server, you'd put localhost there and maybe a couple of internal subnets. In Kubernetes, this list becomes critically important because it must cover the entire internal cluster topology. Miss one suffix, and some inter-service traffic goes through the external proxy.

What Must Be in a Cluster's NO_PROXY

  • localhost and 127.0.0.1 — calls within the pod itself.
  • Pod CIDR range — the subnet from which pod addresses are allocated, e.g., 10.0.0.0/8 or your cluster's specific range.
  • Service CIDR range — the ClusterIP subnet for services, often 10.96.0.0/12.
  • Cluster DNS suffixes — .svc, .svc.cluster.local, .cluster.local. These cover all internal service DNS names.
  • kubernetes.default — the API server name that applications and agents call.
  • Cloud metadata — the address 169.254.169.254, if you're in a cloud environment, so metadata requests don't go through the proxy.

NO_PROXY Syntax Nuances

There's a whole layer of non-obvious gotchas here that even experienced engineers stumble on.

First, different libraries interpret entries differently. Some treat .cluster.local with a leading dot as a suffix and match all subdomains. Others require the format cluster.local without the dot. Practice: specify both variants, with and without the dot, to cover the most clients.

Second, CIDR notation support isn't universal. Go's library understands 10.0.0.0/8, but older versions of some clients in other languages don't — they need individual addresses or ranges in a different format. Test the behavior of your specific stack.

Third, ports. If a NO_PROXY entry is given without a port, it usually applies to any port on that host. But some clients match strictly against the port. With non-standard ports for internal services, this is important to verify.

Insight from practice: nine out of ten incidents of broken internal communication after introducing a proxy are due to an incomplete NO_PROXY. Build a reference list for your cluster once, put it in a shared ConfigMap, and reuse it across all deployments. That saves dozens of hours of debugging.

What Doesn't Pick Up Environment Variables

The naive belief that "I set HTTP_PROXY, so all traffic now goes through the proxy" is doubly wrong in a cluster. There's an entire class of components that simply ignore these variables. You need to know them in advance.

Kubelet and System Components

A container's environment variables are only visible to processes inside that container. Kubelet — the node agent that pulls images, starts containers, and talks to the API server — lives at the node level, not the pod level. It doesn't read env from the manifest. If you need kubelet to pull images through a proxy, the configuration is done at the level of the kubelet systemd service or the container runtime config, not in the pod spec.

# /etc/systemd/system/kubelet.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://gate.proxeon.net:8080"
Environment="HTTPS_PROXY=http://gate.proxeon.net:8080"
Environment="NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,.svc,.cluster.local"

The same applies to the container runtime — containerd or CRI-O. Image pulls go through their own network context. Proxy configuration for the image registry is set in the runtime config, and that's a separate story from pods.

Images With Their Own Configuration

Many popular images have built-in network settings that override environment variables. For example, package managers, web servers, or proxy tools inside the image may read their own config file instead of the environment. If the application inside the container uses, say, a settings file with explicitly specified connection parameters, your env variables will simply go unnoticed.

A separate category is applications where the network client is initialized before the environment is read or caches settings at startup. Changing a variable without restarting the process won't have any effect there.

Individual SDKs and Language Runtimes

This is the trickiest group. Respecting HTTP_PROXY is a convention, not a standard. Some follow it, some don't.

  • Go. The standard http.Client via ProxyFromEnvironment respects the variables. But if the code creates a transport with an explicit nil Proxy, the variables are ignored.
  • Python. The requests library reads the environment by default. But low-level sockets, some gRPC clients, and async libraries don't.
  • Java. The JVM uses its own system properties http.proxyHost and https.proxyHost and by default doesn't read environment variables at all. They need to be passed via JAVA_TOOL_OPTIONS.
  • Node.js. The built-in http module doesn't respect environment variables. You need third-party agents that read them.
  • gRPC. Some implementations read a special grpc_proxy variable rather than the standard ones.

The conclusion is simple: you can't assume that just because a variable is declared, all traffic will go through it. Test each runtime separately. For the JVM, for example, the manifest looks like this:

env:
 - name: JAVA_TOOL_OPTIONS
value: "-Dhttp.proxyHost=gate.proxeon.net -Dhttp.proxyPort=8080 -Dhttps.proxyHost=gate.proxeon.net -Dhttps.proxyPort=8080 -Dhttp.nonProxyHosts=localhost|127.0.0.1|*.svc|*.cluster.local"

Note: in the JVM, the exclusion separator is a vertical bar, not a comma, and the patterns use an asterisk. Another trap for those who mechanically copy NO_PROXY.

Secrets: Proxy Credentials via a Secret, Not in the Manifest

If your proxy requires authentication, the question of storing the login and password comes up. The temptation is strong: paste them right into the URL inside the env value. Don't do it, and here's why.

Deployment manifests almost always live in version control. A plaintext password in env means a credential leak into Git history, accessible to anyone with repo access. Beyond that, env values are visible to anyone who can run kubectl describe pod. That's a direct violation of the principle of least privilege.

The right path is a Secret object. It stores sensitive data separately, with the ability to restrict access via RBAC and enable encryption at the etcd storage level.

apiVersion: v1
kind: Secret
metadata:
 name: proxeon-credentials
type: Opaque
stringData:
 proxy-user: my_account
 proxy-pass: s3cr3t_token_value

Then we wire the values from the secret into the container's environment variables via secretKeyRef, and assemble the proxy URL itself so the credentials don't show up in the manifest:

env:
 - name: PROXY_USER
valueFrom:
 secretKeyRef:
name: proxeon-credentials
key: proxy-user
 - name: PROXY_PASS
valueFrom:
 secretKeyRef:
name: proxeon-credentials
key: proxy-pass
 - name: HTTPS_PROXY
value: "http://$(PROXY_USER):$(PROXY_PASS)@gate.proxeon.net:8080"
 - name: NO_PROXY
value: "localhost,127.0.0.1,10.0.0.0/8,.svc,.cluster.local"

Kubernetes substitutes values from previously declared variables using the dollar-and-parentheses syntax. This lets you assemble a URL with credentials at runtime without putting the password directly in the manifest text. Note: the assembled HTTPS_PROXY value will still be visible in the running container's env, so additionally restrict who can exec and describe these pods.

Best Practices for Working With Secrets

  • Enable etcd encryption at rest — otherwise secrets are merely stored in base64, which isn't protection.
  • Restrict access to secrets via RBAC: a pod should only see its own secret.
  • Rotate proxy credentials regularly and automate pod restarts after rotation.
  • Consider external secret managers connected via a CSI driver, so credentials never land in etcd at all.
  • Never log the assembled proxy URL — the password will leak into your log collection system.

The Sidecar Approach: When It's Justified and What It Delivers

We've already mentioned sidecar as one of the three layers. Now let's examine it as a standalone strategy in more detail, because it's the most flexible tool if you're willing to pay for it in resources.

When a Sidecar Is Justified

  • The application can't read HTTP_PROXY and rewriting its code isn't possible or too expensive.
  • You need a unified proxying policy without editing every application.
  • You need transparent traffic interception, including for non-HTTP protocols.
  • You need observability: egress connection metrics, tracing, auditing.
  • You need retry, timeout, and circuit-breaking policies at the connection level.

What a Sidecar Gives You Beyond Proxying

This is where the real value lies. A sidecar isn't just a packet redirector. Properly configured, it becomes a control and observation point for all of a pod's egress traffic.

  • Metrics. How many requests went out, to which hosts, with what latency, how many errors. Without a sidecar, you'd have to collect this data in each application separately.
  • Tracing. Distributed traces of egress calls, tied to incoming requests.
  • Reliability policies. Automatic retries of idempotent requests, timeouts, concurrent connection limits.
  • Unified TLS. The sidecar can terminate and establish secure connections centrally.
  • Auditing and compliance. A full log of exactly where the application went — invaluable for compliance.

Example Pod With a Sidecar Proxy

Below is a simplified template where the main container routes egress HTTP requests to a local sidecar, which then proxies them onward through Proxeon. For the application, the proxy address is localhost, which automatically excludes inter-service traffic from external proxying if the application talks to its neighbors directly.

apiVersion: v1
kind: Pod
metadata:
 name: app-with-proxy-sidecar
 labels:
app: billing
spec:
 containers:
- name: app
 image: registry.example.com/billing:2.1.0
 env:
- name: HTTP_PROXY
 value: "http://127.0.0.1:3128"
- name: HTTPS_PROXY
 value: "http://127.0.0.1:3128"
- name: NO_PROXY
 value: "localhost,127.0.0.1,10.0.0.0/8,.svc,.cluster.local"
- name: egress-proxy
 image: registry.example.com/egress-agent:1.0.0
 ports:
- containerPort: 3128
 env:
- name: UPSTREAM_PROXY
 value: "http://gate.proxeon.net:8080"
 resources:
requests:
 cpu: 50m
 memory: 64Mi
limits:
 cpu: 200m
 memory: 128Mi

Native Sidecar and Startup Order

The classic sidecar woe is a startup race. If the main application starts up and makes its first egress request before the sidecar is ready to accept connections, the request fails. Starting with Kubernetes 1.28, support for native sidecar containers appeared: they're declared in the initContainers block with restartPolicy Always and are guaranteed to start and stay alive before the main container. This cleanly solves the race problem, without hacks like startup delays and retries.

spec:
 initContainers:
- name: egress-proxy
 image: registry.example.com/egress-agent:1.0.0
 restartPolicy: Always
 ports:
- containerPort: 3128

Egress Gateway: A Stable Egress Address for the Cluster

The third layer deserves its own discussion, because it solves the problem that most often brings people to proxies in a cluster in the first place: a predictable egress IP.

Why You Need a Fixed Egress IP

External partners, payment gateways, and data providers often work off an IP allowlist. They say: we only accept requests from these IPs. In a regular cluster, a pod's egress address is the address of the node the pod happened to land on at that moment. There are many nodes, they scale, get replaced, get added by autoscaling. The address is unpredictable. The partner can't add the entire node pool to the allowlist, especially a pool that changes.

An egress gateway solves this: all egress traffic gathers at a single point with a fixed address. The partner adds one or two stable IPs to the allowlist — and everything works. When using Proxeon as the egress point, you get a stable external address that you hand to partners once.

How Traffic Is Routed to the Egress Gateway

The mechanism depends on the CNI. Some CNI plugins support an egress policy object where you describe: traffic from pods with certain labels going outward should exit through a certain node or address. The plugin configures the corresponding SNAT and routing rules automatically. Conceptually, it looks like this:

apiVersion: policy.example.io/v1
kind: EgressPolicy
metadata:
 name: billing-egress
spec:
 selector:
matchLabels:
 egress: proxeon
 egressIP: 203.0.113.10
 destinationCIDRs:
- 0.0.0.0/0

The exact syntax differs between CNIs, so check your plugin's documentation. The idea is unchanged: label the pods whose traffic goes through the gateway, and set a stable exit address.

Egress Gateway Reliability

Since the gateway is a single point, it's also a single point of failure. Rules for survival:

  • Add redundancy: at least two gateway nodes with automatic failover.
  • Monitor exit availability with a separate synthetic request outbound every minute.
  • Watch throughput: all north-south traffic flows through the gateway, so a bottleneck will affect everything.
  • Separate policies: critical traffic and background traffic are better routed down different paths, so background load doesn't interfere with what matters.

Diagnosing Egress Traffic in the Cluster

When something goes wrong — and it will — you need an arsenal of checks. Let's assemble a practical set of commands and approaches.

Step 1: Find Out the Real Egress IP From a Pod

The first thing we check: what address the pod is visible from to the outside world. Launch an ephemeral debug container or exec into an existing pod and hit a service that returns your public address.

kubectl run nettest --rm -it --image=registry.example.com/nettools:1.0 --restart=Never -- sh
# inside the container
curl -s https://api.ipify.org
curl -s -x http://gate.proxeon.net:8080 https://api.ipify.org

The first request shows the address without a proxy, the second explicitly through the proxy. If both are identical and you expected different, the proxy isn't being applied. If the second returns the expected Proxeon address, the proxy works and the issue is in the application's configuration.

Step 2: Check Whether the Application Sees the Variables

kubectl exec deploy/worker -c app -- env | grep -i proxy

If the output is empty, the variables didn't reach the container. Check the manifest and that the pod was recreated after the change. As a reminder: changing env requires recreating the pod, it's not picked up on the fly.

Step 3: Check DNS Resolution From Inside

Many errors masquerade as proxy problems when it's actually DNS. Check resolution of an internal name and an external one:

kubectl exec deploy/worker -c app -- nslookup orders.default.svc.cluster.local
kubectl exec deploy/worker -c app -- nslookup api.external-partner.com

If the internal name doesn't resolve, the problem is in CoreDNS or the request went to the external proxy because the suffix is missing from NO_PROXY. That's direct confirmation that the exclusion list is incomplete.

Step 4: Where to Look for Failures

  • Application logs. Look for connection errors, timeouts, proxy authentication failures (usually code 407).
  • Sidecar logs. If one exists, you can see which requests it accepted and where it sent them.
  • Egress gateway metrics. Growth in failures and latency at the gateway.
  • Pod events. kubectl describe pod will show startup problems, including the sidecar race.
  • NetworkPolicy. Check whether a network policy is blocking egress traffic — a common cause of silent failures.

Typical Problem Signatures

  • Code 407 from the proxy — wrong or missing credentials. Check the Secret.
  • Timeout when calling an internal service after enabling the proxy — incomplete NO_PROXY.
  • Different egress IPs on repeated requests — traffic isn't going through the egress gateway, regular node SNAT is at work.
  • The first requests after startup fail, then everything works — sidecar race, switch to a native sidecar.
  • A Java application ignores the proxy — you forgot about JAVA_TOOL_OPTIONS and system properties.

Common Mistakes to Avoid

Let's gather in one place the rake-stepping that happens most often. Check yourself against this list before, not after, an incident.

Incomplete NO_PROXY

The absolute leader. You forgot Service CIDR, forgot the .svc suffix, didn't account for the cloud metadata address — and got a cascade of mysterious failures. Always start from a complete reference list for your cluster.

Plaintext Proxy Password in the Manifest

A leak into Git and into describe output. Secrets only, restricted access only.

Assuming All Runtimes Respect the Variables

The JVM, Node.js, and some gRPC clients ignore HTTP_PROXY. Test each stack separately and configure it natively.

Ignoring Kubelet and the Runtime

Image pulls through a proxy are configured at the node level, not the pod level. Don't be surprised images won't pull if you only configured env in the manifest.

Sidecar Startup Race

The application starts before the proxy and drops its first requests. The solution is native sidecar containers.

Egress Gateway Without Redundancy

A single point of failure without duplication takes down all egress traffic. Add redundancy and monitor.

Mixing Up CIDR Formats

You specified 10.0.0.0/8 for a client that doesn't understand CIDR. Check which format your library eats and provide an alternative.

Not Recreating Pods After Changing env

You changed a variable in the deployment, but old pods keep running with old values until restarted. Ensure a rollout happens.

Forgetting Lowercase Variable Names

Some libraries only read http_proxy in lowercase. Duplicate the names in both cases.

Tools and Resources

What to keep handy for working with egress traffic in a cluster.

Diagnostic

  • kubectl exec and kubectl debug — the foundation for checks from inside a pod.
  • Ephemeral debug containers — let you attach a set of network tools to a running pod without rebuilding the image.
  • A network utilities image — curl, dig, nslookup, traceroute, gathered into one container for quick checks.
  • A public IP return service — to verify the real egress address.

Infrastructure

  • A ConfigMap with the reference NO_PROXY — a single source of truth for all deployments.
  • Secret and external secret managers — for proxy credentials.
  • Service mesh — if you need a sidecar with out-of-the-box observability.
  • CNI egress policies — for routing to the gateway.
  • Proxeon (proxeon.net) — proxy infrastructure for a stable egress address and authenticated access.

Monitoring

  • Egress connection metrics from the sidecar or egress gateway.
  • Synthetic availability checks for external addresses from the cluster.
  • Alerts on growth in 407 codes and egress request timeouts.
  • A dashboard with the distribution of egress traffic by destination host.

Case Studies and Results

Let's look at three generalized scenarios showing how the choice of layer affects the outcome.

Case 1: Payment Integration and IP Allowlisting

A team integrated an external payment gateway that only accepts requests from agreed-upon addresses. They first tried environment variables at the container level — and ran into the problem that during autoscaling, pods spread across new nodes with new addresses, and the egress IP still remained the node's address, not the proxy's. Some requests started getting rejected.

The solution: they moved the payment service to egress through Proxeon with a fixed egress address. The partner added one IP to the allowlist. Rejections due to an unknown address disappeared. An additional benefit was a centralized log of all calls to the payment gateway for auditing.

Case 2: Broken Inter-Service Communication After Introducing a Proxy

A company added HTTP_PROXY to all deployments in one go through a shared template. Within minutes, errors flooded in: services stopped seeing each other. Internal calls were going to the external proxy, which couldn't resolve them.

Diagnosis took time until they checked resolution from inside a pod and saw that the .svc.cluster.local name was going to the proxy. The cause: NO_PROXY contained only localhost. They built a complete reference list with Pod CIDR, Service CIDR, and all DNS suffixes, put it in a ConfigMap, and wired it into all pods. Internal communication was restored. Since then, the reference NO_PROXY has been a mandatory part of the deployment template.

Case 3: Egress Traffic Observability via Sidecar

A security requirement: know exactly where each application reaches out to, with a full log. The applications were in different languages, and some couldn't read HTTP_PROXY. Editing the code of dozens of services was too expensive.

They introduced a sidecar proxy into the pod template. Applications route egress traffic to a local agent, which proxies through Proxeon and writes metrics: destination host, latency, response code. A dashboard of egress calls per service appeared. They also configured timeouts and retries at the sidecar level, which reduced the number of cascading failures when external APIs were briefly unavailable. The cost was additional resources for the sidecar, but the gain in observability and reliability justified it.

FAQ: Common Engineer Questions

Why does traffic to a neighboring pod go through the external proxy even though it's obviously an internal address?

Because the HTTP client doesn't know the address is internal. It sees a name or address and, if it's not in NO_PROXY, routes the request to the proxy according to the rules. The client doesn't distinguish east-west from north-south itself — the exclusion list does that for it. Add internal suffixes and subnets to NO_PROXY.

Do I need to configure a proxy for pulling images too?

Yes, but not via pod env. Image pulls are performed by the container runtime and kubelet at the node level. Proxy config for the registry is set in the runtime configuration or the kubelet systemd unit. Container environment variables have no effect on this whatsoever.

What matters more for a stable egress IP — a sidecar or an egress gateway?

The egress gateway. A sidecar proxies an individual pod's traffic, but the egress address is still determined by where the connection goes next. For a guaranteed fixed address that an external partner will accept, you need either an egress gateway or an exit through an external point with a permanent address, for example through Proxeon.

Why does a Java application ignore HTTP_PROXY?

The JVM historically doesn't read environment variables for proxies. It uses its own system properties http.proxyHost, https.proxyHost, and the http.nonProxyHosts exclusions. Pass them via JAVA_TOOL_OPTIONS. And remember: the exclusion separator in the JVM is a vertical bar, and patterns use an asterisk, not CIDR.

How do I store a proxy password securely?

In a Secret object with RBAC-restricted access and etcd encryption enabled. Wire values in via secretKeyRef. Don't write the password in plaintext in the manifest, or it'll leak into Git and describe output. For higher requirements, use an external secret manager via CSI.

I changed the environment variable, but the behavior didn't change — why?

Environment variables are fixed at container startup. Until the pod is recreated, it runs with the old values. Update the deployment so a pod rollout happens. Also check whether the application reads the environment at all, rather than caching settings at initialization.

How do I figure out which NO_PROXY format my library needs?

Empirically: configure the proxy, call an internal name from inside the pod, and see whether the request went to the proxy or direct. If it went to the proxy, the format isn't recognized. Try the variant with and without a leading dot, add individual addresses instead of CIDR, check the case of the variable name. The specific client's documentation is your best guide.

Should I always use a service mesh for proxying?

No. A service mesh is a powerful tool with observability and policies, but it carries noticeable overhead and operational complexity. If the task is only to route egress traffic through a proxy, a lightweight sidecar agent or an egress gateway will be cheaper. Adopt a mesh when you genuinely need its full capabilities.

What about traffic that isn't HTTP at all?

HTTP_PROXY variables only work for HTTP and HTTPS clients that respect them. For arbitrary TCP connections, you need transparent interception at the sidecar or egress gateway level with the appropriate routing rules. Environment variables are powerless here.

Can I set a different proxy policy for different pods?

Yes. At the container level — through different env in different deployments. At the egress level — through label selectors in egress policies, routing labeled pods to the desired exit. Combining them gives flexibility: a baseline exit through the gateway plus individual settings for specific services.

Conclusion: Building a Rollout Checklist

We've traveled from why a "like on a regular server" setup doesn't work in a cluster, to the three layers of embedding a proxy, NO_PROXY subtleties, secrets, sidecars, egress gateways, and diagnostics. The main takeaway: in Kubernetes, egress traffic isn't one variable but a layered system, where what matters most isn't how to enable the proxy but how not to break internal communication with it.

Let's assemble a final rollout checklist worth keeping in front of you at every deployment.

Cluster Proxy Rollout Checklist

  • Chose the layer. Consciously selected container, sidecar, or egress gateway for the specific task.
  • Built a complete NO_PROXY.

    About the Author

    Andrey Kokh

    Andrey Kokh

    Leading Expert and Business Consultant

    Work Experience: Leading expert with 12 years of experience. Consults Forbes-listed companies, author of 3 books. Teaches at HSE and SKOLKOVO. His methodologies are used by hundreds of companies across Russia. RBC and Forbes expert on strategic development and digital transformation.
    Education: Higher School of Economics. Faculty of Economics, Master's Program
    Expertise:
    Strategic Consulting Digital Transformation Change Management Business Strategy Innovation Management Organizational Development Lean Management Agile Transformation

    Share this article: