Streaming

GET /v1/searches//events: the SSE wire format, and how to resume a dropped stream.

GET /v1/searches/{id}/events
Authorization: Bearer <credential>

Ordinary server-sent events, Content-Type: text/event-stream. The credential is resolved before the first byte, so a bad credential is a 401 and a run that is not yours is a 404, both as normal JSON responses. Once the stream has started there is no way back to a status code, which is why every refusal happens up front.

The wire format

id: 41
data: {"type": "status", "stage": "search", "round": 0, "index": 0, "message": "Pre-discovery added 3 remembered entities to the universe."}

: keepalive

id: 42
data: {"type": "progress", "have": 4, "target": 50, "round": 0, "leads": []}

Three rules cover it:

• One frame is an id: line, a data: line, and a blank line. The data: payload is always a single line of JSON, and the event’s own kind is the type field inside it. There is no SSE event: name to switch on.
• A line beginning with : is a keepalive. It carries nothing and exists only so an idle proxy does not drop the connection during a long quiet stage. Ignore it; do not try to parse it, and do not treat it as progress.
• id: is the event’s sequence number. They start at 1 and increase monotonically for the life of the run. This is the whole of what makes resumption work. Do not assume there are no gaps: an event too large to store is dropped and the sequence carries on past it, so track the last id: you actually received rather than counting.

The stream ends, cleanly and with no closing frame of its own, once the run is terminal and you have read everything. An ended stream is therefore not evidence that the search succeeded, only that there is nothing more to read. The run’s status is the authority. See The run.

EventSource cannot be used. It has no way to set an Authorization header, and this endpoint has no other way to take a credential. Read the stream with fetch and a stream reader in a browser, or an ordinary streaming HTTP client elsewhere, and handle the reconnect yourself as below.

Resuming without a gap and without a repeat

A long run will lose its connection. There are two obvious recoveries and both are wrong:

• Reconnect and replay from the start. Every event you already handled arrives again. A UI duplicates rows and cards; a script double-counts.
• Reconnect and take it from here. Everything emitted while you were away is gone, silently. Nothing reports a gap, so the run looks like it simply had less to say.

The cursor is the fix. Keep the last id: you saw and send it back:

How	Value
Last-Event-ID: 41 header	Resume after event 41. Preferred: it is the header a reconnecting client sends naturally.
?since=41 query parameter	The same thing, for a client that cannot set the header.

Both mean “send me events with a sequence number greater than 41”. If you send both, the header wins. If the header is not a number the server falls back to since rather than refusing the stream. since must be zero or more; a negative value is a 422. A cursor at or past the end of the run returns an empty stream, not an error, so there is no harm in resuming a run that has already finished.

lastSeq on the run object is the highest sequence number written so far, so it is also a valid cursor. That is what to use when you want the tail of a run you have never streamed.

A reconnect loop that is correct

import json, os, time, requests

BASE = "https://eniac.floworks.ai"
CREDENTIAL = os.environ["ENIAC_CREDENTIAL"]
AUTH = {"Authorization": f"Bearer {CREDENTIAL}"}
TERMINAL = {"done", "stopped", "budget_exhausted", "interrupted", "error"}

def stream(run_id: str):
    """Every event of one run, exactly once, across any number of drops."""
    last_seq = 0
    while True:
        headers = dict(AUTH)
        if last_seq:
            headers["Last-Event-ID"] = str(last_seq)
        try:
            with requests.get(
                f"{BASE}/v1/searches/{run_id}/events",
                headers=headers, stream=True, timeout=(10, 120),
            ) as response:
                response.raise_for_status()
                seq = None
                for line in response.iter_lines(decode_unicode=True):
                    if not line or line.startswith(":"):
                        continue                       # blank line or keepalive
                    if line.startswith("id: "):
                        seq = int(line[4:])
                    elif line.startswith("data: "):
                        yield json.loads(line[6:])
                        if seq is not None:
                            last_seq = seq             # advance only once handled
        except requests.RequestException:
            pass                                       # a drop is ordinary

        # The stream also ends normally when the run is terminal, so ask the run
        # instead of reconnecting for ever. raise_for_status is outside the try
        # on purpose: a 401 or a 404 is not something to reconnect through.
        run = requests.get(f"{BASE}/v1/searches/{run_id}", headers=AUTH, timeout=30)
        run.raise_for_status()
        if run.json()["status"] in TERMINAL:
            return
        time.sleep(1)                                  # do not spin on a flapping link

for event in stream("6f1c0c5b9a1e4b7b8d2f3a4c5e6d7f80"):
    if event["type"] == "final":
        rows = event["table"]
    elif event["type"] == "error":
        print("failed:", event["message"])

Two details in there are the ones worth copying. last_seq advances only after the event has been handled, so a crash mid-event replays that one event rather than skipping it. And the loop asks the run for its status instead of assuming the stream ending means the run ended, because those are different facts.

Events live for 24 hours. The event log is a transport, not the record: each event expires a day after it was written. The stream and the result field on a run are both read from it, so collect what you need within 24 hours of the run finishing. Nothing else about the run expires.

Starting a stream late

Nothing is lost by connecting after the POST. The run writes its events whether or not anyone is reading, so a stream opened a minute in with no cursor replays the run from event 1 and then continues live. That is also how a second reader works: any number of readers can stream the same run at once, each with its own cursor.