# Infinite Canvas

Native and WebAssembly Raylib 6 prototype for a world-space, immediate-mode
infinite canvas.

## Build and run

```bash
# Native development build with Raygui controls
bazel run //infinite_canvas:dev

# Start directly in the ten-browser stress scene
INFINITE_CANVAS_BROWSER_STRESS=1 bazel run //infinite_canvas:dev

# Native build without development UI
bazel run //infinite_canvas:app

# Browser build and local server
bazel build //infinite_canvas:infinite_canvas_wasm
bazel run //infinite_canvas:serve
```

Open `http://127.0.0.1:4343/index.html` after starting the server. Pass a
different port as an argument only when needed.

The native development target opens at 1920x1200 and starts with a gallery
layout containing one non-overlapping example of every entity type.

Theme mode defaults to `auto`, which follows the OS light/dark preference at
startup. Rendering consumes a shared semantic `Canvas_Theme` palette for the
canvas, developer controls, and camera-pinned notifications. Until the planned
config file lands, `INFINITE_CANVAS_THEME=light|dark|auto` provides an explicit
override. The developer panel also includes a live light/dark mode toggle.

## Controls

- Middle-mouse drag, or Space + left-mouse drag: pan
- Mouse wheel: scroll vertically
- Ctrl/Cmd + mouse wheel: zoom around the cursor
- WASD or arrow keys: pan (disabled while a text area has keyboard focus)
- Left-mouse drag on an entity: select and move it
- `P`: pin or unpin the selected entity in screen space so it remains visible
  while the camera pans and zooms; pinned entities show a Lucide pin glyph
- `M`: show centered live dictation text and toggle microphone capture
- `Enter`: send visible dictation text to the spatial agent
- `Ctrl/Cmd+Enter`: submit the selected text area's thought to Copilot
- `Delete`/`Backspace`: fade out and remove the selected entity
- Development toolbar: spawn/clear entities, reset the camera, or restore the
  deterministic demo scene; spawned entities fade in and Clear fades all
  entities out before removal. Use the header arrow to collapse the toolbar
  into a small restore button.
- `View camera context`: open a live, scrollable snapshot of entities
  intersecting the camera viewport and their current semantic values
- `Lucide icon gallery`: create or revisit a dedicated canvas region containing
  the complete searchable, scrollable Lucide 1.31.0 icon set and fit the camera
  to it
- `10 browser stress scene`: lay out ten compact independent browser views for
  performance testing
- `White`/`Grid`: toggle the optional grid; the default canvas is pure white

Canvas primitives include shapes, lines, text, buttons, editable text areas,
dropdowns, accordions, cards, calendars, switches, tables, notifications,
scrollable areas, images, and web content. Accordions
expand with eased height, chevron, divider, and content transitions; cards
toggle emphasis, calendars select dates, and switches animate their track color
and thumb position when toggled. The default browser build embeds
`https://mrjunejune.com` in a camera-tracked iframe. Native builds open
`https://www.google.com` in a CEF off-screen browser whose pixel buffer is
uploaded into a Raylib texture. It remains inside the canvas and participates
in pan, zoom, clipping, and entity dragging on Windows, macOS, and Linux. Web
views show a Raylib-rendered Lucide back/forward toolbar and current URL only
while selected. The toolbar eases open by pushing the page viewport downward
and reverses the transition when deselected. Click the URL field to edit it;
Enter navigates, Escape cancels, and standard Ctrl/Cmd clipboard and selection
shortcuts work. Drag the remaining outer edge to move the view and drag the
bottom-right handle to resize it. Embedded pages inherit the selected canvas
light or dark color scheme where the browser backend supports it. Multiple web entities have independent
browser instances, URLs, focus, input, and rendering. Browser resources are
assigned only to visible entities, off-screen views are suspended, and the
native CEF pool supports up to twelve live browsers. Small background views
render at a reduced frame rate, larger views receive a moderate frame budget,
and the focused view runs at 30 FPS so scenes with ten compact browsers remain
responsive without giving every page the maximum rendering budget. Native
browser viewport and texture resolution follow the displayed view size in
64-pixel buckets, capped at 1280x800, so resized views become sharper without
reallocating continuously during a drag.

The Lucide gallery is one scalable Raylib-rendered canvas entity rather than
thousands of individual scene entities or an embedded browser. Reopening it
reuses the existing entity and moves the camera back to its fitted bounds. Its
ISC-licensed vector data is generated from Lucide 1.31.0 and the visible grid
rows are drawn directly with Raylib primitives.

## Spatial agent sessions

Authenticate the Copilot CLI once:

```bash
bazel run //mrjunejune:run_inference_stack -- --authenticate
```

Then start dictation, the Copilot LiteLLM gateway, and the canvas with one
target:

```bash
bazel run //infinite_canvas:orchestration_dev
```

`//infinite_canvas:agent_dev` remains an alias. Closing the canvas stops the
supervised local services.

Press `M` to start or stop the local WebRTC microphone session. The browser
transport is prewarmed and remains hidden: it acquires the native microphone
permission, device, signaling, ICE connection, and transcript data channel
ahead of the first hotkey while keeping the audio track disabled. `M` therefore
only unmutes an already-ready channel, so speech from the beginning of the
utterance is retained. Partial and final speech appear directly as minimal centered text
inside a retained `Dictation` text-area entity spawned near the camera center.
The entity word-wraps, scrolls to the latest line, and can be dragged by its
header to any position on the board. The same scratchpad is reused for every
thought and clears after submission; it does not create duplicate dictation
windows. A small pulsing `Listening` indicator
remains in the bottom-left while capture is active. In-progress speech uses
muted text; `Backspace` clears the entire current thought without stopping
capture. Pausing with `M` and resuming appends to the same thought rather than
replacing it. Press `Enter` to commit and send the current utterance while the
microphone stays live for the next thought; press `M` or `Escape` to stop it.
For typed input, select a text area and press `Ctrl/Cmd+Enter`.
Native CEF grants audio only to the fixed loopback dictation transport; it does
not show an inaccessible browser permission prompt or grant microphone access
to normal canvas web entities.

The Copilot SDK receives the visible camera context, including existing
conversation entity IDs, through the shared `canvas_orchestrator` prompt
profile. It returns a routing decision to append to an existing session or
create a new one. Requests run off the render thread through the shared
`Inference_Bridge`. Conversation cards retain
their own title, transcript, working state, scroll position, selection, and
pinning state, forming the first rich-context container for future typed image
and component resources. Runtime configuration and Copilot token paths come
from the ignored `mrjunejune/.config`; no separate canvas environment file is
required. The Dictation scratchpad is excluded from camera context because its
text is submitted separately, and only visible conversation cards are eligible
append targets. Conversation cards follow the latest appended message and can
be collapsed from their header. The archive icon moves a session to the
off-canvas parking lot, excluding it from context; `Parking lot (N)` in
Developer Controls jumps there, and the restore icon returns the session to
its previous board position. Collapse and expansion use the same eased retained
animation model as accordion entities rather than snapping between heights.
Canvas startup pre-creates only the persistent orchestration SDK session,
moving its setup out of the first submitted thought without eagerly creating
worker sessions.

Developer Controls can save and load named native `.zmap` snapshots. The
versioned binary representation restores camera position, zoom, z-order, and
retained entity state with checksum and bounds validation. Agents may also
return a bounded `entities` array to showcase an answer directly with native
calendar, card, table, text, image, or browser entities.

Agent submissions initially show only a small pulsing canvas icon. The
orchestrator chooses `action` presentation for simple one-off work, replacing
the icon with the result and creating no chat/session card, or `conversation`
presentation for work that benefits from retained discussion. The icon has an
animated hover label and can be dragged to choose the result location.

Tables support row selection, and notifications behave as bottom-right toast
cards: multiple toasts overlap compactly, fan into a spaced stack on hover,
and slide/fade away from either Undo or close. Position, size, stack expansion,
and dismissal all use eased frame-rate-independent transitions. Scrollable areas consume the
mouse wheel without zooming the camera. Image
entities render HTTPS URLs or native filesystem paths through CEF; browser
builds render URLs or paths available in the Emscripten filesystem. Override
the demo image in native development with
`INFINITE_CANVAS_IMAGE_SOURCE=/path/to/image.png`.

The first native build downloads the platform CEF 151 minimal distribution.
Linux stages its runtime once under
`~/.cache/zenbu/infinite-canvas/cef-151`; application state and the RGBA paint
buffer remain owned by the Dowa arena, while CEF owns its browser internals.

Buttons animate on hover/press and toggle when clicked. Dropdowns open a real
option menu with hover and selection states. Selected text areas accept typing,
wrap text, support click/drag selection and scrolling, and provide
Ctrl/Cmd+A/C/X/V clipboard shortcuts plus Shift selection, word navigation,
Home/End, Backspace, and Delete. Drag a text area's narrow outer padding to
move it without changing the selection.

Camera context is derived from the visible world rather than a hidden
conversation store. Panning entities outside the viewport removes them;
zooming out includes and concatenates newly visible entities. Each entry
contains identity, type, placement, pin state, and type-specific state such as
text content, dropdown selection, switch value, selected calendar date or table
row, scroll position, image source, and browser URL. The current text format is
intentionally provisional so it can later be replaced by the final API
contract.

The prototype uses the Inter variable font under the SIL Open Font License.

Agent-oriented architecture and extension notes live in
[`docs/README.md`](docs/README.md).

## Memory ownership

The application creates one Dowa arena. Application state and the fixed-capacity
entity array are arena-owned, there are no per-frame first-party allocations,
and the arena is released once during native shutdown. Browser lifetime memory
is reclaimed with the WebAssembly runtime when the page closes.