Authentication & Passwords
Connect’s auth model has two layers, and confusion between them is the single most common source of “Session requires password” errors. This page describes both layers and the cache that ties them together.
The two layers
| Layer | What it authenticates | Where it lives |
|---|---|---|
| Workspace credential | ”Is this caller allowed to talk to the relay at all?” | API key (ck_…) or OAuth token. Held by the workspace. |
| Machine password | ”Once a session is opened on this machine, is the operator allowed to drive it?” | Per-machine setting on the agent. Stored locally in internal/security/machinepw. |
A workspace credential is mandatory. A machine password is optional — only machines that explicitly set one require it. Both must be satisfied before a streaming session opens.
The workspace key gets you to the relay. The machine password gets you into the session. They are two different gates, configured in two different places, even though Connect collapses them into one flow at attach time.
Streaming vs unary auth
The relay exposes two RPC shapes, and they handle authentication
differently. Mixing them up is the canonical source of pain. The
authoritative reference is internal/connect/client/CLAUDE.md
§streaming vs unary auth; the short version:
Streaming (ConnectTerminal bidi)
Used by AttachStream (interactive PTY) and ExecOnce (one-shot
recovery door-opener). The auth gate is inline in the stream:
client → RegisterRequest
server → AuthChallenge(methods=["password"])
client → AuthResponse(password=...)
server → AuthSuccess(session_token=...)Once the client receives AuthSuccess, the session_token is
cached by sessionauth.Default() for 24 hours.
Unary (SendInput, SendResize, Execute, …)
Unary RPCs do not speak password. The server’s @session_guard
decorator only checks for an x-session-token metadata header
against Redis. If the header is missing or unknown, the call fails
with:
UNAUTHENTICATED: Session requires password.
Authenticate via ConnectTerminal first.This is the canonical “I’m seeing the password error” trap. The fix is not to install another password somewhere — it is to make sure a streaming auth has run for that session ID before any unary RPC fires.
Client.authCtxForSession(ctx, sessionID) is the helper that
attaches the cached token to a unary call. Every unary RPC against
a session must go through it.
The session token cache
internal/connect/sessionauth/store.go is a small process-wide
singleton:
Put(sessionID, token)is called by the streaming attach handler onAuthSuccess.Get(sessionID)is called byauthCtxForSessionto attachx-session-tokenmetadata to unary RPCs.- TTL is 24 hours. Expired entries are pruned on access.
- Memory-only — nothing lands on disk.
Practical consequences:
- Inside one daemon process, the cache is shared across
surfaces. A streaming attach by
cmdop connectpopulates the token; subsequent unary calls fromcmdop connect execor theconnecttoolagent tool reuse it. - Across daemon restarts, the cache is gone. First call after restart pays the streaming-open cost again.
- Across hosts, the cache is independent. Each machine’s daemon caches its own tokens.
Password sources, in order
When a streaming attach gets an AuthChallenge, the client tries
password sources in this order:
--passwordflag. Explicit per-call value.- Local store.
internal/security/machinepwkeeps a per-machine password record locally. Lookup key is the machine ID. Set viacmdop connect password set, cleared viacmdop connect password clear. CMDOP_AGENT_PASSWORDenv var. Single value, same for every machine. Used by automation and CI.- TTY prompt. If a human is at the terminal, the CLI prompts inline.
The agent-tool surface skips step 4 — bots cannot answer prompts.
If steps 1–3 do not yield a value, the call fails with an
auth_error.
# Set a password locally for one machine.
cmdop connect password set vps-audi
# Password: <typed in>
# Inspect (only that a value is set; never prints the password).
cmdop connect password get vps-audi
# Remove.
cmdop connect password clear vps-audiThe CMDOP_AGENT_PASSWORD env var applies to every password
challenge in the same process. For multi-machine fan-outs where
machines have different passwords, prefer the local store (per-machine)
or pass --password explicitly per call.
ExecOnce recovery (no extra prompts)
cmdop connect exec is unary. Without help, it would fail
immediately on a password-protected machine because no prior
streaming auth has populated the cache. The CLI handles this
transparently with the ExecOnce recovery path:
- Try unary
Execute. - On
ErrSessionTokenRequired, open a brief streamingConnectTerminalsession, answer theAuthChallenge, capture thesession_token, store it insessionauth.Default(). - Retry the unary
Execute— this time it succeeds.
Non-password machines pay zero overhead (step 1 just works). Password machines pay one extra streaming open per process. After that, every unary against the same machine is fast.
The full incident history is documented in
internal/connect/client/CLAUDE.md §ExecOnce recovery.
Workspace credential failures
The other half of auth — the workspace key — fails earlier and with clearer errors. Common ones:
| Error | Cause | Fix |
|---|---|---|
no API key for workspace "production" | Resolver chain bottomed out. | cmdop connect key set <key> or cmdop login. |
unauthenticated: invalid api key | Key is wrong or revoked. | Rotate the key in the dashboard. |
unauthenticated: oauth token expired | OAuth token past 72h, no refresh. | cmdop login again. |
permission_denied: machine X not in workspace Y | You switched workspaces; machine belongs to the previous one. | cmdop connect workspace use <correct>. |
For the full workspace credential precedence, see credential-resolver.
MFA and OAuth
MFA lives on the OAuth login flow, not on Connect itself. When you
run cmdop login, the browser-based flow handles whatever
multi-factor your account requires. The OAuth token that comes back
is the workspace credential — Connect never sees the password,
TOTP, or webauthn assertion directly.
OAuth tokens have a 72-hour lifespan with refresh. The CLI refreshes
silently while it can; if the refresh fails, you re-run cmdop login. There is no “MFA prompt” in cmdop connect itself.
Talking to a password-protected machine
Putting it together, here is what happens for a password-protected machine on its first call after daemon restart:
cmdop connect exec prod-api-1 -- uptime- Resolver picks
prod-api-1from the workspace. - Workspace credential resolves (flag → env → workspace → OAuth).
- Unary
Executefires. Server returnsErrSessionTokenRequired. - CLI opens streaming
ConnectTerminal. Server sendsAuthChallenge. - CLI tries the four password sources. Finds the password in the local store.
- Server returns
AuthSuccess(session_token=...). CLI caches it. - CLI retries unary
Execute, this time withx-session-tokenmetadata. Server is happy. uptimeruns and returns.
Subsequent calls in the same process skip steps 4–7 entirely.
Self-to-self machines
When the caller and target share the same OAuth identity (verified
via CallerHostname), the permission gate is bypassed (see
server-to-server). The password gate is
not — even on self-to-self, a password-protected machine still
demands its password. The two gates are independent.
Common errors
| Symptom | Layer | Cause | Fix |
|---|---|---|---|
Session requires password. Authenticate via ConnectTerminal first. | Unary | No streaming auth has populated the cache for this session. | The CLI’s ExecOnce path handles this — if you see it from a custom integration, ensure you call streaming attach first. |
auth_error: no password source | Streaming | All four password sources empty. | Set the password locally or via env; or pass --password. |
unauthenticated: invalid session token | Unary | Cached token expired (24h passed) or evicted. | Just rerun the call — ExecOnce re-primes the cache. |
permission_denied | Workspace | API key valid, but doesn’t cover this machine. | Switch workspace or use a key that does. |
Related
The workspace-credential side of auth, in full.
Where streaming auth is most visible.
Unary auth and the ExecOnce recovery path.
The credential origin and how to switch between workspaces.