# Migrating A Critical Kubernetes Deployment From The Default Namespace Without Any Downtime

Somewhere in your cluster there's probably a deployment sitting in the `default` namespace that everyone knows shouldn't be there. Nobody put it there maliciously, it just happened, early on, before anyone had opinions about namespace hygiene, and now half your other services quietly depend on it. Moving it is now a tricky problem.

That was the exact situation with a service I'll call `auth-svc`: a authenticaion service that dozens of other services called constantly, sitting in `default` for years, and about to become a genuine problem the moment it needed namespace-scoped things, its own ingress rules, its own policies, that `default` structurally couldn't give it. Moving it wasn't optional forever. But it also couldn't go down, not even for a few seconds. This wasn't a vague "other services might complain" risk: `auth-svc` handled authentication for that entire region's cluster, so if it went down, nobody in that region could log in. Full stop.

Before getting into why this is actually hard, it's worth being precise about what "moving it" means. There are two completely separate paths into `auth-svc`, and both have to keep working throughout the move, or fixing one just creates an outage in the other. Everything inside the cluster reaches it the ordinary way: other services resolve `auth-svc.default.svc.cluster.local` through Kubernetes' own internal DNS and get routed to a pod, the standard Service mechanism. Everything outside the cluster reaches it through an ingress instead, a completely separate mechanism that has nothing to do with that DNS name. Whatever the fix turned out to be, it had to solve for both paths, not just the one that's easier to reason about.

## Why "just move it" doesn't work

The obvious plan, move the deployment, update the references, done, falls apart the moment you look at who's actually calling this thing. As mentioned, dozens of other services reference `auth-svc` by its cluster-internal DNS name, owned by different teams, on different release cycles. There's no atomic moment where you flip a switch and every one of them simultaneously starts using a new name. Some team's service hasn't been redeployed in months. You shouldn't be coordinating that.

The tooling got in the way too. Our deploy pipeline only knew how to ship a service to one namespace. There was no "deploy this to two places at once" option, and modifying the shared pipeline logic every other team also depended on felt like exactly the kind of blast radius we didn't want to introduce. Whatever the fix was, it had to fit inside a single-namespace deploy, not require rewriting shared infrastructure.

On top of that, we had an OPA policy which enforced that identical ingress rules couldn't exist live in two namespaces at once, a sane rule that exists specifically to stop the kind of half-finished migration that leaves routing ambiguous.

## The insight: a forwarding address

The piece that made this solvable: I didn't need to migrate every consumer's understanding of where `auth-svc` lives. I needed to migrate the service, and quietly redirect anyone still asking for the old address.

[Kubernetes has exactly this mechanism,](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) and it's easy to forget it exists because you almost never need it: an `ExternalName` service. Instead of pointing at pods, it points at another DNS name, functioning essentially like a CNAME (see [my DNS article](https://downtherabithole.dev/how-dns-really-works) if you want to learn more about how that works). Deploy the real thing at its new home, then convert the old Service object into a forwarding address:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: auth-svc
  namespace: default
spec:
  type: ExternalName
  externalName: auth-svc.identity.svc.cluster.local
```

Every consumer still calling `auth-svc.default.svc.cluster.local` gets silently redirected to the real thing in its new namespace. Nobody changes a line of code on their end. It's the same trick as a postal forwarding order: you don't visit every person who might send you mail and update their address book, you tell the post office where you actually live now, and everything gets redirected until people eventually update it themselves, at their own pace, with zero coordination required on your part.

That forwarding trick only earns its keep if you actually confirm it's working before you lean on it. Once the proxy was live, the next step wasn't scaling anything down, it was watching the metrics through the crossover: checking that traffic hitting the old address was genuinely landing on the new deployment, not silently failing or looping somewhere. Only once that looked clean did the old pods get scaled to zero rather than deleted outright. Scaling to zero costs nothing and buys an instant rollback, just scale back up, if anything downstream looked wrong later. Deleting them outright would have meant rebuilding from scratch if something went sideways, so there was no reason to give up that safety net early. We could defer the cleanup to a later point in time.

![](https://cdn.hashnode.com/uploads/covers/698ce732e249cba68b3bbdb5/ea94010c-d633-4ee9-9680-088cffad7f5f.png align="center")

## The chicken-and-egg problem

The remaining wrinkle was the ingress. External traffic to `auth-svc` doesn't come in through the DNS-based Service mechanism at all, it comes in through an ingress, and I needed a working ingress in the new namespace before I could safely remove the one in the old namespace. But the policy engine wouldn't allow both to exist at once; identical ingress rules across two namespaces is exactly the ambiguous state it exists to prevent.

Classic chicken-and-egg: can't create the new one without a policy exception, can't delete the old one first without a traffic gap.

The fix was a temporary, explicit exception rather than fighting the policy itself: annotate the new namespace to bypass the duplicate-ingress check just for this migration, stand up the new ingress alongside the old one for a short overlap window, confirm traffic was flowing correctly to the new deployment, then delete the old ingress and let the exception age out. A brief, deliberate window where both existed, rather than a gap where neither did.

The patch would look something like this:

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: identity
  annotations:
    policy.example.com/allow-duplicate-ingress: "true"
```

## How it actually shipped

None of this went straight to production. It ran in dev first, then a staging cutover, with a couple of weeks between the staging success and doing it for real, mostly just to sit with it and see whether anything subtle showed up under real traffic before betting a critical path on it.

The production cutover itself ended up being the boring part. All the actual difficulty was front-loaded into getting the design right. Once the plan was solid, executing it was closer to a formality than an event.

## Why this matters beyond one service

Here's the thing about `default` namespace sprawl: it's rarely caused by carelessness. It's caused by the complete absence of pressure to ever fix it. Someone creates a policy restricting new services from landing in `default`, good practice, but nobody sets a deadline or a plan for the services already there. They just sit. For years, in this case. Nothing forces the issue until a team needs something namespace-scoped that `default` structurally can't give them, and only then does the debt come due.

If you're staring at a similarly stuck service, the pattern generalises past this one migration. An `ExternalName` proxy buys you a zero-coordination path to move anything addressed by DNS, provided you're willing to hold two versions in careful overlap for a short, deliberate window rather than trying to cut everything over at once. The next time someone tells you a service can't be moved because too many things depend on it, that's usually a sign nobody's looked for the DNS-shaped seam it can be split along.

* * *

*This only works cleanly because everything here talked to* `auth-svc` *through its DNS name rather than a hardcoded IP or ClusterIP. If you've got consumers that skip DNS entirely, and some legacy systems do, you're solving a different, uglier problem.*