Mercurial

comparison infinite_canvas/docs/agent-sessions.md @ 281:c57149ad216e default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Copilot-Session: f68442b1-fa8f-46a0-9689-81710613bbd4
author MrJuneJune <me@mrjunejune.com>
date Tue, 18 Aug 2026 22:18:15 -0700
parents 49e9e591c9bb
children
comparison
equal deleted inserted replaced
280:49e9e591c9bb 281:c57149ad216e
22 the current utterance finalizes without closing the microphone session. 22 the current utterance finalizes without closing the microphone session.
23 5. `agent_service_copilot.c` sends the new thought plus 23 5. `agent_service_copilot.c` sends the new thought plus
24 `Canvas_Scene_Build_Visible_Context()` through the shared asynchronous 24 `Canvas_Scene_Build_Visible_Context()` through the shared asynchronous
25 `Inference_Bridge`. 25 `Inference_Bridge`.
26 6. The Bazel-managed Copilot SDK sidecar uses the `canvas_orchestrator` profile 26 6. The Bazel-managed Copilot SDK sidecar uses the `canvas_orchestrator` profile
27 and returns JSON choosing `create` or `append` and a conversation entity ID. 27 and returns JSON choosing `action` or `conversation` presentation,
28 The canvas materializes that decision as a retained rich conversation card. 28 `create` or `append` routing, a conversation entity ID, and optional typed
29 showcase entities. The canvas materializes only the requested surface.
30
31 All submissions begin as one small pulsing Raylib/Lucide indicator anchored to
32 the board. Hovering reveals its working label, and dragging it changes the
33 world position where an eventual result or conversation is materialized. A
34 one-off action replaces it with the resulting entity or compact notification
35 and does not create another agent/session. Complex work creates a retained
36 conversation card after orchestration completes; continuations append to an
37 already-visible conversation.
29 38
30 The scratchpad is excluded from serialized camera context because the submitted 39 The scratchpad is excluded from serialized camera context because the submitted
31 thought is already sent separately. Only conversation entities currently in 40 thought is already sent separately. Only conversation entities currently in
32 view are valid append targets; moving a conversation out of view removes it 41 view are valid append targets; moving a conversation out of view removes it
33 from the orchestrator's routing choices. 42 from the orchestrator's routing choices.
39 Raylib. The sidecar maintains the persistent 48 Raylib. The sidecar maintains the persistent
40 `infinite-canvas-orchestrator` SDK session and uses the loopback LiteLLM 49 `infinite-canvas-orchestrator` SDK session and uses the loopback LiteLLM
41 gateway for authenticated GitHub Copilot inference. Only one orchestration 50 gateway for authenticated GitHub Copilot inference. Only one orchestration
42 request is admitted at a time in this prototype. 51 request is admitted at a time in this prototype.
43 52
44 `agent_service_copilot.c` warms that orchestration session plus three reserved 53 `agent_service_copilot.c` warms only the persistent orchestration session
45 worker sessions during initialization. They share the same 54 during initialization. This moves Copilot SDK client/session setup ahead of the
46 `canvas_orchestrator` profile and move Copilot SDK client/session setup ahead of 55 first user turn without creating unused worker sessions. The orchestrator is
47 the first user turn. The persistent orchestrator is resumed when available; 56 resumed when available.
48 reserved workers are created fresh because they have no conversation history
49 to recover, avoiding expected `session.resume` errors during startup.
50 57
51 ## Conversation entities 58 ## Conversation entities
52 59
53 `CANVAS_ENTITY_CONVERSATION` is intentionally more than an editable text area. 60 `CANVAS_ENTITY_CONVERSATION` is intentionally more than an editable text area.
54 It retains: 61 It retains:
76 - `//infinite_canvas:orchestration_dev` owns the dictation server, LiteLLM 83 - `//infinite_canvas:orchestration_dev` owns the dictation server, LiteLLM
77 gateway, and canvas lifecycle. `//infinite_canvas:agent_dev` is an alias. 84 gateway, and canvas lifecycle. `//infinite_canvas:agent_dev` is an alias.
78 - One persistent Dictation scratchpad is reused and cleared after submission. 85 - One persistent Dictation scratchpad is reused and cleared after submission.
79 Submitted turns are retained in conversation entities after orchestration. 86 Submitted turns are retained in conversation entities after orchestration.
80 - One Copilot request runs at a time. 87 - One Copilot request runs at a time.
88 - Simple `action` presentation stays in the primary orchestrator and does not
89 create a conversation card, worker, or sub-session.
90 - Agent-created entities pass through a strict type allowlist and count bound.
91 Browser and image entities accept only absolute HTTP(S) URLs.
81 - Conversation transcript storage is currently bounded by 92 - Conversation transcript storage is currently bounded by
82 `CANVAS_ENTITY_TEXT_CAPACITY`. 93 `CANVAS_ENTITY_TEXT_CAPACITY`.
83 - Linux uses the Copilot SDK bridge. Other platforms currently select an 94 - Linux uses the Copilot SDK bridge. Other platforms currently select an
84 explicit unavailable stub. 95 explicit unavailable stub.