Throttling and the circuit breaker

Why this lives on disk, not in memory

An agent calls roost as a fresh process, one invocation per shell-out. If the politeness throttle were an in-memory timer, it would reset to zero on every single call — a loop that calls roost search ten times in a row would never see its own history. So the throttle state lives on disk, in $XDG_STATE_HOME/roost/ratelimit.json (override the directory with ROOST_STATE_DIR), and every invocation of the google backend reads and updates it. That’s what makes the spacing real across a process-per-call agent loop instead of a no-op.

This only applies to the google backend. serpapi is a paid, keyed API with its own rate limits on the provider side, so roost’s local throttle exempts it entirely.

The min interval and the burst of 2

Two numbers govern spacing, both in throttle.py:

• DEFAULT_MIN_INTERVAL — 6.0 seconds between requests to Google.
• DEFAULT_BURST — 2 requests allowed back-to-back before that spacing kicks in.

The burst isn’t a convenience — it’s a floor. One logical search costs two HTTP requests: first resolve_place() fetches /travel/hotels/<slug> to resolve the location to a place id (mid), then a second fetch of the same path with the built ts= descriptor gets results. That’s exactly what a person clicking into a city and reading results does. Without a burst allowance, the very first search for a city roost hasn’t seen before would fail its own throttle on the second request, before any result ever came back.

The place cache makes repeat searches cheap

Resolved place ids are cached on disk at $XDG_STATE_HOME/roost/places.json for 30 days. The first search for a city costs two requests (resolve + fetch); every search after that for the same location costs one, because resolve_place() returns the cached mid without a network call:

def resolve_place(location, *, wait, max_wait, no_throttle):
    if mid := ts.cached_mid(location):
        return mid, s
    body = _fetch_google(...)   # only reached on a cache miss

If you’re driving roost across many searches for the same market — comparing dates or brands for one city — the cache is why the second and later calls run at the full 6-second cadence instead of needing double that.

The circuit breaker

The throttle’s second job is recognizing when Google has actually pushed back, and stopping harder than a spacing delay. Any HTTP 429, a 403 or 503, or a 200 response that looks like a challenge page opens the breaker via record_block(). Its cooldown follows a fixed backoff schedule, indexed by how many blocks have happened in a row without a clean response in between:

[30, 60, 120, 300, 600, 1800]  # seconds

First block: 30s. Second consecutive block: 60s. Up through a 30-minute cooldown at the top of the schedule. A clean response resets the counter (record_success()) — the schedule only climbs on repeated failures, not on total requests. While the breaker is open, roost doctor skips its live backend probe rather than sending a request that would just deepen the block.

Fail-fast is the default, and why

By default, if a request would need to wait — either the min-interval hasn’t elapsed with no burst allowance left, or the circuit breaker is open — roost does not sleep. It raises an error immediately:

• Min-interval not satisfied → exit 7, RATE_LIMITED, with retryAfterSeconds telling you exactly how long to wait.
• Circuit breaker open → exit 20, BLOCKED, with the same.

The reasoning is in the source directly: “a hung CLI deadlocks an agent loop.” An agent that shells out to roost and blocks on a sleeping subprocess has no way to do anything else in the meantime — no timeout of its own, no visibility into why it’s stalled. Getting a structured error back immediately, with the wait time in the payload, lets the agent decide what to do next: back off, try a different query, or surface the wait to whatever is above it.

{
  "error": "throttled locally to stay polite; 4s until the next request is allowed",
  "code": "RATE_LIMITED",
  "remediation": "wait 4s and retry, or pass --wait --max-wait 4"
}

When to use --wait

--wait opts into blocking sleep, up to a --max-wait cap (default 30.0 seconds). If the remaining wait — either the min-interval gap or the circuit-breaker cooldown — fits under --max-wait, roost sleeps that long and then proceeds. If it doesn’t fit, you still get the fail-fast error.

Reach for --wait when you’re driving roost as a person at a terminal, or from a script where a few extra seconds of latency is fine and you’d rather not hand-write a retry loop. Leave it off — the default — inside an agent loop that has its own scheduling or retry logic, since that’s exactly the deadlock scenario the fail-fast default exists to avoid.

Why not just pass --no-throttle

--no-throttle skips the local politeness layer entirely — no spacing, no circuit-breaker check. It exists mainly for tests and debugging. Using it against the live google backend defeats the thing the throttle is protecting: roost is a keyless scraper hitting Google’s own hotel search pages, and the circuit breaker’s backoff schedule is what keeps a burst of retries from escalating into a longer block. Skipping it doesn’t make Google respond faster — it just removes roost’s own signal for when to stop, and a real 429 or challenge page still gets recorded as a block afterward regardless of whether local throttling was on.

If you’re hitting rate limits routinely, the fix is --backend serpapi (a real paid API with its own limits, exempt from this local throttle) — not --no-throttle.

Inspecting throttle state

roost doctor reports live throttle state as part of its checks, without touching the network in a way that could deepen an open block:

roost doctor --json

{
  "ok": true,
  "checks": [
    { "name": "backend:google", "ok": true, "detail": "google backend ready (keyless)" },
    {
      "name": "throttle",
      "ok": true,
      "detail": "closed; 3 request(s) in the last 10 min",
      "state": {
        "backend": "google",
        "lastRequest": 1755791234.12,
        "blocked": false,
        "blockedUntil": null,
        "cooldownSeconds": 0,
        "consecutiveBlocks": 0,
        "recentRequests": 3
      }
    }
  ]
}

When the breaker is open, checks[].detail for throttle reads something like circuit breaker OPEN — 47s remaining (2 consecutive blocks), and ok flips to false. The same snapshot — under the throttle key — is also embedded in roost schema --json, so an agent can check cooldown state without a dedicated doctor call.

Tip

roost doctor is safe to poll before a batch of searches. It reads throttle state from disk; it doesn’t make a request against Google while the circuit breaker is open.

See roost doctor and the exit codes reference for the full 7 vs. 20 vs. 8 distinction, and legitimacy for why the breaker is a stop signal rather than something to route around.