How it works

roost has no official API to call. On the default google backend it fetches a real Google Hotels page and reads it the way a browser would, then turns whatever comes back into typed JSON or a structured error. This page walks the request from a location string to a parsed card, and the two failure modes that matter most: a page that no longer parses, and a page that isn’t really a results page at all.

Step 1: resolve the place

Google Hotels keys a search to a place, not a string. Every city has a Freebase-style id that looks like /m/0dclf (for Kyoto), and that id — not the city name — is what goes into the search descriptor.

roost resolves it by fetching the plain city page:

GET https://www.google.com/travel/hotels/<slug>?hl=en

<slug> is the location, lowercased and space-joined with + (ts.slug). The response HTML is scanned for every substring matching /m/[0-9a-z_]{2,12}, and the id winning that page is whichever one appears five or more times — a real place id saturates every card link and descriptor on the page, while a stray unrelated reference shows up once or twice. If nothing clears that bar, roost raises NOT_FOUND (exit 5) rather than guessing.

The resolved id is cached at $XDG_STATE_HOME/roost/places.json for 30 days (_CACHE_TTL), keyed by the slug. A repeat search for the same city skips this fetch entirely and goes straight to step 2 — which is also why the throttle needs a burst allowance of 2: the first search for an uncached city legitimately costs two requests (resolve, then fetch), and without that headroom it would fail its own politeness check.

Step 2: build the `ts` descriptor

Dates, guest count, and currency are packed into a single base64-encoded protobuf, passed as the ts query parameter on the same /travel/hotels/<slug> path. This is the part roost got right where a prior-art library got it wrong, so it’s worth being precise about.

fast-hotels, the library roost mined for its access-path mechanics, encodes this parameter as ths and writes dates into it as strings. Google accepts that request, returns HTTP 200, and renders a page of real-looking hotel cards — for its own default date window, not the one requested. There is no error, no warning, nothing to catch in a status code. It’s the worst failure shape an availability tool can have: confidently wrong dates.

roost avoided this by not trusting the mined encoder’s guess about the schema. Instead the real Google Hotels UI was driven directly and the ts parameter it produced was decoded field-by-field. The verified layout, from ts.py:

ts {
  1: 1                                     # descriptor version
  2: { 1:{1:3}, 1:{1:3}, 2:0 }             # result-mode flags
  3: {                                     # the search
    1: { 2: { 1: <place mid>, 7: <place name> }, 3: {} }
    2: { 2: { 1: Date(check-in), 2: Date(check-out), 3: 1 }, 6: { 1: <adults> } }
  }
  5: { 1: { 7: <currency> }, 3: {} }
}
Date { 1: year, 2: month, 3: day }

The load-bearing difference is that Date submessage: check-in and check-out are each a nested {year, month, day} message of three varints, not a string. roost ships its own minimal protobuf wire encoder for this (ts.py’s _varint/_tag/_msg/_date helpers) — a dozen fields don’t justify a full protobuf runtime dependency, and keeping the encoder self-contained keeps --help and schema fast since nothing here needs to be imported eagerly.

ts.build() assembles the descriptor from a resolved place id and name, the stay dates, adult count, and currency, base64-urlsafe-encodes it, and strips the = padding. The final request for a dated search is:

GET https://www.google.com/travel/hotels/<slug>?ts=<descriptor>&hl=en&curr=<currency>

Both this fetch and the place-resolution fetch in step 1 go through one choke point, backend.http_get, which checks the path against a robots.txt-derived allowlist before issuing anything — /travel/hotels/<city> is allowed; /travel/search, /travel/entity, and the clk/rpc/stories paths are not, and roost never touches them.

Step 3: parse the response — and only there

Everything roost knows about the shape of a Google Hotels page lives in one file, parse.py. No other module parses upstream HTML. That isolation is deliberate: when Google reshapes the page, the fix is contained to this one adapter, and the failure surfaces as a typed error instead of a crash or a silently empty result.

Result cards are matched with div.uaTTDe (CARD_SELECTOR) and property names with h2.BgYkof (NAME_SELECTOR). Each card’s text is flattened into one pipe-separated run — roughly:

Sakura Cross Hotel | GREAT DEAL | $92 | $92 nightly | $304 total |
3 nights with taxes + fees | 60% less than usual | 4.9 | (2.5K) | …

and a set of regexes pull the structured fields out of that run: nightly and total price (matched separately — Google states both explicitly, so roost reports both and infers neither), star class, rating and review count, deal percentage, taxes-included, and the amenity list. A card that yields no name, or neither a nightly nor a total price, is treated as an ad or placeholder and dropped rather than emitted as a malformed property.

When parsing fails: exit 21, not a crash

parse_cards raises ParseError in exactly two cases: the card selector matches nothing at all, or it matches cards but none of them yield both a name and a price. backend.search catches that ParseError and turns it into schema_drift(...) — exit code 21. The message an agent sees names the exact CSS selector that stopped matching, because when the real fix is “Google changed this page and roost needs an update,” retrying is never the right response — 21 exists specifically so an agent doesn’t loop on it.

That’s the “upstream changed shape” branch. The other branch — “we got blocked” — is exit 20, and telling them apart is where the parse order matters.

Detecting a challenge served as HTTP 200

A soft block from Google doesn’t come back as a 403 or a 429 (those are handled directly as BLOCKED too, but they’re unambiguous). The trickier case is a page that returns HTTP 200 and is not the results page it claims to be — a CAPTCHA or “unusual traffic” interstitial dressed up as a normal response.

There’s a trap here: a genuinely successful Google Hotels page contains the substrings captcha and recaptcha several times, inside its own JS bundles. Matching those unconditionally would flag every good response as a block and wedge the circuit breaker permanently — this was caught in implementation and is pinned by a regression test (test_good_page_is_not_flagged_as_blocked).

parse.py resolves this with two tiers, checked at two different times:

Strong signals (looks_blocked) — phrases that only ever appear on a real interstitial: "our systems have detected unusual traffic", "unusual traffic from your computer", "/sorry/index", "please click here if you are not redirected". These are checked before parsing, on every response, because they’re unambiguous — a genuine results page never contains them.
Weak signals (looks_blocked_weak) — "recaptcha", "captcha", "verify you are human", "are you a robot", "automated queries", "not a robot". These are ambiguous on their own (they show up in good pages’ JS too), so they’re consulted only after parsing has already found zero result cards. At that point the page has already failed to look like results by every other measure, and the weak signal is what decides why: present → BLOCKED (exit 20, back off); absent → SCHEMA_DRIFT (exit 21, the parser needs an update).

A separate check, looks_consent, catches Google’s EU consent interstitial (consent.google.com or "before you continue to google") and reports it as its own CONSENT_REQUIRED error, since results never render behind that page at all — no amount of waiting or retrying gets past it, only a non-EU locale or --backend serpapi.

Every one of these checks — strong signals, the consent gate, and the weak signals after a failed parse — records a block against the shared circuit breaker so the process backs off rather than hammering a page that’s already rejecting it. See politeness for how that state persists across processes.

Why this shape holds up

The pattern across every layer here is the same: keep the part of roost that has to guess about Google’s internals (the place-id heuristic, the ts field layout, the CSS selectors, the block-signal lists) as small and isolated as possible, and make every failure in that guess come back as a typed, documented outcome — NOT_FOUND, SCHEMA_DRIFT, BLOCKED — never a stack trace and never a wrong answer presented as a right one. See exit codes for the full table and the read-only boundary for what roost deliberately does not do with this access.