Scale & restart

Three commands cover most lifecycle work: scale, restart, stop. They wrap the same underlying reconciler, so they're safe to chain.

Immediate scale

rune scale api 5

The reconciler creates or destroys instances until 5 are running and healthy. The CLI blocks until rollout completes (or the timeout — --timeout=5m by default).

Detach (fire-and-forget — don't wait for the rollout):

rune scale api 5 -d

Gradual scale

When ramping a stateful or memory-hungry service, step up:

rune scale api 10 --mode=gradual --step=2 --interval=30s

This adds 2 instances every 30 seconds until you hit 10. If any step fails health checks, the operation aborts.

Restart

restart replaces every instance in place — the server stamps a new template generation and the reconciler swaps the instances, so the desired scale never dips through zero:

rune restart api

Use it after editing a configmap or rotating a secret — mounted files don't hot-reload, but a restart picks them up. Restarting a stopped service brings it back at its last non-zero scale.

Restarts are rolling by default — see Rolling updates below.

Rolling updates

Changing a service's spec — a new image, new env, a rotated secret — no longer takes the whole service down. Rune starts a replacement, waits until it's serving, then retires an old instance, and repeats. The same applies to rune restart.

The default behaviour, in one paragraph. Rune starts one replacement, waits until it's serving, retires the oldest instance, and repeats until every replica runs the new spec. Old instances keep taking traffic the whole time. Each one gets drainSeconds (default 5) to finish in-flight requests after it stops receiving new connections. Services that can't run two copies of a replica update one at a time instead. Want the old stop-everything behaviour? updateStrategy: recreate.

The two knobs

service:
  name: api
  image: ghcr.io/acme/api:v2
  scale: 4
  drainSeconds: 10          # shutdown grace, applies to EVERY teardown
  updateStrategy: recreate  # optional; `rolling` is the default

drainSeconds is how long an instance keeps serving in-flight work after it leaves the load-balancer rotation, before it gets SIGTERM. It governs every teardown — scale-down, stop, deletion, a health-triggered restart — not just updates. Setting it to 0 still leaves a one-second floor, because withdrawal takes a moment to propagate.

updateStrategy: recreate restores the pre-rolling behaviour: every instance goes down before any replacement starts. Use it when a service genuinely cannot run two copies — an exclusive external lock, a single-writer migration, a licence-seat limit.

That's the whole configuration surface. Everything else, Rune derives per service — but derived doesn't mean hidden, and it's worth knowing what to expect:

What that looks like

A scale: 4 service updating, counting instances that exist and instances actually serving:

Note the direction: it goes up to 5, not down to 3. The spare is created first, and an old instance is retired only once the replacement is serving. Serving never drops below 4. Seeing five instances of a four-replica service mid-deploy is normal — that's the spare, and it disappears when the update converges.

A service that can't run a spare goes 4 → 3 → 4 instead: it has to retire before it can create, so it dips by one. Unavoidable when only one copy of a replica can exist at a time.

Each replica costs three passes — create, wait, retire — so a scale: 4 service converges in about twelve, and a scale: 7 in about twenty-one.

Most of these aren't settings, because each has one right answer given what Rune already knows about the service. One is: how much availability you're willing to trade for a faster deploy.

Deploying a large service faster

The default replaces one replica at a time, which gets slow as scale grows — a scale: 7 service takes around twenty-one passes. If you're willing to run at 3/7 while deploying, say so:

updateStrategy:
  type: rolling
  minServing: 3

Now Rune retires up to four at once. Same service, same deploy:

Five passes instead of twenty-one, with serving touching exactly the 3 you asked for and no lower. Omit minServing and nothing changes.

The same trade at scale: 4 with minServing: 2 converges in four passes instead of twelve.

This is the one value Rune can't work out for you. Whether a service can run a spare copy is a fact about the service; whether 3 of 7 is enough depends on your traffic and your SLO, and nothing in the spec implies it.

Two rules are checked when you cast:

• minServing can't exceed scale — no update could satisfy it.
• On a service that can't run a spare copy, minServing can't equal scale. That combination has no room to start a replacement and no allowance to retire one, so an update could never begin. Rune rejects it and tells you which resource is responsible.

What you can't set: how many extra copies

There's no maxSurge. Rune runs at most one extra instance per service during an update, and that isn't adjustable.

The reason is that on one to three boxes, spare capacity is the scarce thing — a host is usually sized for about the sum of your scales. Kubernetes exposes maxSurge because a cluster can normally find another node's worth of room; here, the knob that costs nothing is dipping, so that's the one you get. If you need a deploy to go faster, minServing is the lever.

If you need to opt out of rolling entirely, that's updateStrategy: recreate.

Add a readiness probe

Without one, Rune advances the update when the container starts, not when your app is serving, and falls back to a flat five-second wait per replacement. With one, updates are both safer and roughly three times faster:

  health:
    readiness:
      type: http
      path: /healthz
      port: 8080

rune lint warns when an exposed service is missing one.

Services that can't run two copies

Rune detects these — you don't declare it. A service updates one replica at a time if it has a per-replica claimTemplate volume, binds a hostPort, or runs on the process runtime. At scale: 1 that means a brief gap on every update, which is unavoidable when only one copy can exist at a time.

Single-replica workers: read this before upgrading

A scale: 1 service with no volume and no hostPort can run two copies — so during an update, the old and new copy briefly overlap. Before rolling updates, that never happened.

If your service must never overlap with itself — a queue consumer, a cron-style worker, a migration runner — set updateStrategy: recreate on it. rune lint flags exactly these services.

Watching an update

rune status grows an UPDATE column while anything is updating. For detail:

rune describe service api

Update    2/4 replaced · 4/4 serving, 38s elapsed — waiting for replacements to become ready

rune get events -n prod records the lifecycle after the fact: when the update started, which instances were retired and why, and whether it completed or stalled.

When an update stalls

If an update makes no progress for ten minutes, Rune marks the service Failed with reason UpdateStalled. Your old instances are usually still serving — rolling updates retire an old instance only once a replacement is ready, so a bad image typically means the update never got anywhere, not that the service went down.

To diagnose:

rune describe service api      # the Update line says what it was waiting for
rune get events -n prod        # which instance failed, and why
rune logs api --previous       # the failed replacement's own output

The usual cause is a replacement that starts and immediately exits. Fix the spec and rune cast again.

Deploys take longer now

That's the trade: a scale-4 service that used to be replaced in one shot now converges over several reconcile passes. With a readiness probe, expect roughly a minute; without one, longer. If your CI wraps rune cast, make sure its timeout allows for it — rune cast --timeout defaults to 15 minutes.

Restarting instances stuck in create

An instance that never got a container — an unpullable image, a rejected registry credential — exhausts its create attempts and freezes in Stalled; the reconciler stops retrying it on purpose. restart re-arms those instances by clearing the attempt counter and the retry backoff. Fix the cause first, then rune restart api.

If the CLI reports that instances "never got a container and remain stalled", work through the two causes it names — the usual one is that the underlying problem is still there, so rune describe service api has the reason.

Stop

rune stop api

Stops without deleting — desired scale becomes 0, the spec stays. Bring it back with rune scale api N or another cast.

Failure handling

By default, scale operations roll back on health failure. To pin the new scale even if some instances fail to come up:

rune scale api 10 --rollback-on-fail=false

Use this only when you know the service is fine and you'd rather see partial progress than revert.

Inspecting a scale operation

rune get service api -o yaml | grep -A5 status

Status fields you'll see during a scale:

• desiredReplicas — what you asked for.
• readyReplicas — instances passing readiness.
• currentReplicas — instances that exist (may not be ready).
• lastScaleAt, lastScaleReason.

Common patterns

# Drain a service for maintenance
rune stop api
 
# Bring it back to 3
rune scale api 3
 
# Restart after rotating a secret
rune restart api
 
# Slow ramp of a worker pool
rune scale workers 50 --mode=gradual --step=5 --interval=1m

Anti-patterns

• Scaling instead of restarting. rune scale api 0 && rune scale api 3 works but is awkward — use rune restart api, which also re-arms a Stalled instance that never got a container.
• Forgetting -d/--detach in CI scripts when you don't actually want to block.
• Setting --rollback-on-fail=false by default to "make scaling faster." If health is flapping, you want the rollback. Fix the probes.