Streaming Responses

Streaming in SimpleFunctions

Some operations take time — a thesis evaluation might need to process multiple signals, query market data, and run LLM inference. Streaming lets you see results as they're generated rather than waiting for the complete response.

Where Streaming is Used

1. Thesis evaluation: As the agent processes each signal and updates each node, you see results incrementally
2. Chat mode: When conversing with the agent (sf chat), responses stream token-by-token
3. Market scans: Results appear as each market is processed, rather than all at once
4. What-if analysis: Cascade effects are shown as they're computed

Technical Implementation

SimpleFunctions uses Server-Sent Events (SSE) for streaming:

• The client opens a persistent HTTP connection
• The server sends events as they become available
• Each event contains a chunk of the response
• The connection closes when the response is complete

Benefits of Streaming

• Faster perceived latency: You see the first results in <1 second even if the full response takes 10 seconds
• Interruptible: You can cancel mid-stream if you see what you need
• Progress visibility: Know that the system is working, not hung

CLI Streaming

The CLI streams by default for interactive commands. You can disable it for scripting:

sf thesis evaluate --no-stream  # Waits for complete response
sf thesis evaluate              # Streams results as they arrive