view wiki/README.md @ 279:b3b547563ec7

Add Google connector service and agent wiki Implement the C/Seobeo Google Drive and Gmail connector with encrypted OAuth storage, Zenbu authentication, browser testing, AI tool discovery, chunked HTTP decoding, and Bazel coverage. Consolidate repository guidance into progressive wiki documentation and enforce arena-first allocation for new first-party C code. Co-authored-by: Copilot <[email protected]> Copilot-Session: 84c338fd-0939-4bb3-b7f3-1062eb213e5d
author MrJuneJune <me@mrjunejune.com>
date Mon, 17 Aug 2026 22:22:36 -0700
parents
children
line wrap: on
line source

---
title: Zenbu repository wiki
status: canonical
audience:
  - humans
  - AI agents
last_reviewed: 2026-08-17
---

# Zenbu repository wiki

This page is the monorepo knowledge router. It explains what Zenbu contains and
directs humans and agents to the smallest canonical documentation set needed
for a task.

## Reading protocol

```text
repository index -> relevant project wiki -> BUILD graph -> source and tests
```

Do not scan every project document by default. Choose documentation from the
routing table below. For cross-project work, inspect Bazel dependencies first,
then read only the wikis owned by actual producers and consumers.

## Documentation routing

| Task paths or topic | Canonical documentation |
| --- | --- |
| `connectors/`, Google OAuth, Drive, Gmail, AI context tools | [`connectors/wiki/README.md`](../connectors/wiki/README.md) |
| `design_system/`, shared components, tokens, HTML authoring | [`design_system/wiki/README.md`](../design_system/wiki/README.md) |
| `seobeo/`, HTTP, TLS, servers, clients, SSE, WebSockets | [`seobeo/README.md`](../seobeo/README.md), then only the relevant page in [`seobeo/docs/`](../seobeo/docs/) |
| `mrjunejune/`, personal site, auth UI, conversations, media | [`mrjunejune/README.md`](../mrjunejune/README.md) |
| `hg-web/`, Mercurial repository browser | [`hg-web/README.md`](../hg-web/README.md) |
| `dowa/`, arenas, strings, arrays, hash maps, math | [`dowa/README.md`](../dowa/README.md) |
| `deita/`, SQLite wrapper | [`deita/README.md`](../deita/README.md) |
| `s3/`, presigned uploads | Source header and `BUILD`; no dedicated wiki yet |
| `markdown_converter/` | Package `BUILD`, public headers, and tests |
| `gui_ze/`, asset bundling macros | [`gui_ze/wiki/README.md`](../gui_ze/wiki/README.md) |
| Experiments and standalone prototypes | The package README when present, then its `BUILD` |

## Monorepo map

### Shared foundations

- `dowa`: core C types, arena allocation, strings, dynamic arrays/hash maps,
  and math.
- `deita`: SQLite connection, prepared query, and result-set wrapper.
- `seobeo`: networking, HTTP client/server, TLS, SSE, and WebSockets.
- `auth`: Zenbu users, sessions, cryptography, and reusable HTTP auth.
- `config`: Bazel platform configuration.

### Services and applications

- `connectors`: Google data connector service and AI tool boundary.
- `mrjunejune`: personal website and production server.
- `hg-web`: Mercurial repository browser.
- `npc`: MCP-oriented C server experiment.
- `dictation`, `schwab_trader`, `infinite_canvas`, and other package roots:
  focused applications or prototypes.

### Web and asset systems

- `design_system`: shared light-DOM Web Components and design tokens.
- `gui_ze`: Bazel macros for asset copying, bundling, and transformations.
- `assets`: shared icons and other reusable assets.
- `markdown_converter`: C/WASM markdown conversion.
- `rich_editor` and `react_games`: browser bundles consumed by applications.

## Build and workflow

Zenbu is a Mercurial repository. Use:

```sh
hg status
hg diff
```

### Bazel-only rule

Zenbu is Bazel-only. Bazel is not just the C build tool; it is the repository's
execution and dependency boundary.

Use Bazel for:

- C/C++ libraries, binaries, and tests;
- shell and Python entry points;
- Node/Bun-driven frontend generation;
- WASM compilation;
- asset copying, image conversion, and bundles;
- downloaded tools, browser runtimes, and model runtimes;
- integration and end-to-end tests.

Do not add or rely on:

- Makefiles or direct `make` workflows;
- CMake project files;
- manual `cc`, `clang`, or `gcc` build commands;
- `npm run`, `bun run`, or `pip install` as the normal repository interface;
- undocumented system packages or `/usr/local` libraries;
- scripts that bypass Bazel runfiles and declared `data`.

An upstream ecosystem command may run inside a Bazel rule or a narrowly scoped
maintenance workflow, but the committed user/agent interface must be a Bazel
target.

Build and test from the repository root:

```sh
bazel build //seobeo:seobeo
bazel test //seobeo/tests:seobeo_http_framing_test

bazel build //mrjunejune:mrjunejune_server
bazel test //mrjunejune/test:integration_test

bazel build //connectors:connector_server
bazel test //connectors:connector_tests

bazel build //hg-web:hg_web_server
```

Inspect a package's `BUILD` file before broad searches. Keep dependencies on the
smallest target that needs them.

## Package selection guide

Choose an existing first-party package before writing a new helper or directly
depending on third-party code.

| Need | Use | Typical Bazel label | Do not substitute by default |
| --- | --- | --- | --- |
| Integer aliases, booleans, arenas, strings, arrays, hash maps, JSON helpers, math | `dowa` | `//dowa:dowa` | libc allocation scattered through request code, a second containers library |
| SQLite connections and prepared queries | `deita` | `//deita:deita` | direct `sqlite3_*` calls in application packages |
| TCP/TLS primitives without HTTP | `seobeo_min` | `//seobeo:seobeo_min` | raw sockets/OpenSSL setup duplicated in applications |
| HTTP server without WebSockets | `seobeo_tcp_server` | `//seobeo:seobeo_tcp_server` | another embedded HTTP server |
| HTTP server with WebSockets | `seobeo_tcp_server_ws` | `//seobeo:seobeo_tcp_server_ws` | direct WebSocket framing |
| Outbound HTTP/TLS only | `seobeo_tcp_client` or compatibility alias `seobeo_client` | `//seobeo:seobeo_tcp_client` | libcurl subprocesses or hand-built HTTP clients |
| Full HTTP/TLS/WebSocket stack | `seobeo` | `//seobeo:seobeo` | depending on every network source manually |
| Verbose Seobeo diagnostics | `seobeo_debug` | `//seobeo:seobeo_debug` | permanent `printf` debugging |
| User, password, session, cookie, CSRF, and auth crypto | `auth` | `//auth:auth`, `//auth:auth_http` | creating another login/session store |
| Google OAuth, Drive, Gmail, encrypted provider credentials, AI connector tools | `connectors` | `//connectors:connector_core`, `//connectors:connector_service_lib` | provider token logic inside unrelated applications |
| S3 signing and presigned uploads | `s3` | `//s3:s3` | ad-hoc AWS signature code |
| Native and WASM markdown conversion | `markdown_converter` | `//markdown_converter:markdown_to_html_c`, `//markdown_converter:markdown_to_html_wasm` | a second markdown pipeline |
| Shared Web Components, tokens, and UI primitives | `design_system` | package targets in `//design_system` | application-local copies of shared controls |
| Shared icons and generated image assets | `assets` | `//assets:icons` | copying asset files between applications |
| Web asset moves, bundles, WebP conversion, Bun actions, macOS bundles | `gui_ze` macros | `//gui_ze:gui_ze.bzl` | custom copy scripts and undeclared output folders |
| Shared rich text editor browser bundle | `rich_editor` | package targets in `//rich_editor` | forking editor code into an app |
| Personal site server and its production bundle | `mrjunejune` | `//mrjunejune:mrjunejune_server` | treating site-specific code as a generic library |
| Mercurial browser service | `hg-web` | `//hg-web:hg_web_server` | shelling out from unrelated UI code without its API boundary |
| MCP-style C HTTP server experiment | `npc` | `//npc:npc` | assuming it is a general connector framework |
| Raylib application rules | `third_party/raylib` macro plus first-party app package | `//third_party/raylib:raylib.bzl` | manual platform linker flags |
| Qwen/llama.cpp runtime orchestration | `qwen3_vl` | `//qwen3_vl:model`, `//qwen3_vl:serve` | untracked local model commands |

### Package boundary rules

- Put reusable behavior in the narrowest shared package that owns the concept.
- Applications may compose libraries; libraries must not depend on applications.
- Avoid circular ownership such as `dowa` knowing about Seobeo or a site.
- Expose public headers through the owning library target.
- Add deps to the smallest target that compiles the source using them.
- Use platform-aware aliases already provided by packages such as Seobeo and S3.
- Search for an existing helper or macro before creating a sibling implementation.

## Third-party dependency policy

Third-party code enters Zenbu through one of two Bazel-controlled paths:

1. A pinned Bzlmod dependency or repository rule in `MODULE.bazel`.
2. A vendored wrapper under `third_party/<name>/BUILD`.

Applications should depend on public Bazel labels, not include vendored source
paths or reproduce platform flags.

### Prefer first-party wrappers

- Use Deita instead of direct SQLite APIs unless changing Deita itself.
- Use Seobeo instead of raw sockets, OpenSSL transport code, or curl processes.
- Use the S3 package instead of implementing AWS signing in an application.
- Use `gui_ze` macros instead of manual generated-asset copying.
- Use `markdown_converter` rather than adding another markdown dependency.
- Use `auth` for Zenbu identity; external OAuth credentials belong in
  `connectors`, not a parallel user system.

### Adding an external dependency

Before adding one:

1. Search first-party packages and existing `third_party/` wrappers.
2. Confirm the behavior cannot be implemented safely with an existing library.
3. Add the dependency through `MODULE.bazel` when Bzlmod support is suitable;
   otherwise add a focused `third_party/<name>/BUILD` wrapper.
4. Pin versions and SHA-256 hashes for downloaded archives/files.
5. Preserve license files and upstream notices.
6. Expose only the smallest useful Bazel target and visibility.
7. Keep platform selection inside the wrapper, not every consumer.
8. Add or update lockfiles when the ecosystem requires them.
9. Document why the dependency exists and which first-party package owns its
   abstraction boundary.

Do not fetch executable code at ordinary runtime when it can be a declared
Bazel dependency or data artifact.

### Existing external systems

| External system | Bazel integration and intended use |
| --- | --- |
| OpenSSL | Bzlmod `@openssl`; TLS and crypto, normally behind Seobeo/auth/connectors |
| SQLite | `//third_party/sqlite3`; normally consumed through Deita |
| Emscripten | `@emsdk` local override; WASM targets such as markdown conversion |
| Node/npm | rules_nodejs and aspect_rules_js; locked frontend/test dependencies |
| Python/pip | rules_python toolchain and package-specific locked hubs |
| Bun | pinned platform archives and `gui_ze` actions |
| Chromium/Playwright | pinned browser runtime for browser and integration tests |
| Tectonic | pinned runtime for LaTeX rendering |
| FFmpeg | `//third_party/ffmpeg` wrapper for media processing |
| Raylib | `//third_party/raylib` plus `raylib_binary` for native/web applications |
| libuv | `//third_party/libuv` for event-loop experiments such as Postdog |
| LuaJIT | `//third_party/luajit` for packages that explicitly declare it |
| Mercurial | wheel/runtime wrappers for hg-web tooling |
| CEF | repository rule under `third_party/cef` for the infinite-canvas native surface |
| llama.cpp/CUDA runtime | pinned archives exposed through `qwen3_vl` Bazel targets |

## First-party conventions

- Prefer Dowa integer and boolean aliases plus `TRUE`/`FALSE`.
- Allocation is arena-first. New first-party C code must not call raw
  `malloc`, `calloc`, `realloc`, or `free`.
- Create a `Dowa_Arena` at the owner boundary, pass it into helpers, and use
  `Dowa_Arena_Allocate`, arena string helpers, and the `_Arena` array/hash-map
  macros.
- Free each locally owned arena exactly once with `Dowa_Arena_Free` on every
  return path. Never individually free an arena-owned pointer.
- Borrowed request arenas are never freed by callees.
- Long-lived objects should own an arena whose lifetime matches the object.
  A non-arena allocation is permitted only at a true allocator/runtime or
  external-API ownership boundary, must be narrowly documented, and must use
  the matching first-party/external destructor. It is not a convenience escape
  hatch.
- Use Seobeo logging and response-map conventions in Seobeo services.
- Keep runtime secrets in one ignored, documented service config file.
- Commit placeholder templates only; do not commit real credentials.
- Preserve platform-aware Bazel aliases and precise dependencies.

### Arena enforcement

Run:

```sh
bazel test //:arena_policy_test
```

The policy target fails when:

- a new first-party C/H file introduces raw allocation calls; or
- a legacy file increases its reviewed raw-allocation baseline.

The baseline is technical debt, not permission to add more allocation in those
files. Changes should reduce it. Updating the baseline upward requires an
explicitly documented allocator/runtime boundary and review.

Typical ownership:

```c
Dowa_Arena *p_arena = Dowa_Arena_Create(64 * 1024);
if (!p_arena)
  return FALSE;

char *copy = Dowa_Arena_Allocate(p_arena, length + 1);
if (!copy)
{
  Dowa_Arena_Free(p_arena);
  return FALSE;
}

/* All arena-owned values die together. */
Dowa_Arena_Free(p_arena);
```

## Wiki model

The repository index is the schema and router. Project wikis are synthesized,
canonical knowledge for a bounded domain. Source code, `BUILD` files, tests,
and committed config templates are evidence.

When a task changes knowledge:

1. Update the owning project wiki.
2. Update this routing page only if ownership, package boundaries, or canonical
   documentation locations changed.
3. Do not create one-off top-level notes or duplicate normative instructions.
4. Split a project wiki only when its index becomes hard to navigate; every new
   page must be linked from the project wiki.

## Current wiki coverage

| Domain | State |
| --- | --- |
| Repository map and agent routing | Canonical |
| Connectors and AI retrieval | Canonical |
| Design system | Canonical multi-page wiki |
| Seobeo | README plus focused protocol docs |
| Other packages | Existing README or source-first; promote to a wiki when the domain accumulates durable operational knowledge |