# Branching AI chat app — architecture & build plan Conversation branching modeled as a git-like tree, with an infinite canvas for navigation. ## The core data model Every message in the app is a node with a single pointer back to its parent. The "original" conversation is just the chain of nodes from the root down. Branching from any answer means: create a new node whose parent is that answer, instead of appending to whatever was previously the "latest" node. There's no special branching logic to invent — it's the same structure as a linked list, except a node can have more than one child. The canvas is nothing more than a visual layout of that same tree. It doesn't hold any independent truth — it's a rendering of the parent/child relationships already sitting in the database. ```mermaid flowchart TD A[Root message] --> B[Message 2] B --> C["Message 3<br/>Branch point"] C --> D[Branch A] C --> E[Branch B] ``` *Gray-equivalent nodes are the untouched original thread; the branch point and everything that grows out of it forms the new branches.* The one wrinkle worth being deliberate about: when the user sends a message on Branch A, what do you send to the LLM as context? The answer is: walk up the parent chain from Branch A back to the root, collecting every message along the way, and send that as the `messages` array. It's the same operation as `git log` on a branch — you're resolving history by following parent pointers, not storing a separate copy of the conversation per branch. Each node only ever needs to store its own content plus a parent ID; nothing gets duplicated. ## System architecture Four pieces, and only one of them talks to the outside world: ```mermaid flowchart TD Client["Client<br/>Canvas + chat UI"] --> Backend["Backend API<br/>Core orchestration"] Backend --> DB[("Database<br/>Stores the tree")] Backend --> Adapter["LLM adapter<br/>Swap providers"] Adapter --> Anthropic["Anthropic API"] ``` *Everything except the Anthropic API box is a component you build; Anthropic's API is the one external piece, tucked behind an adapter so it's replaceable later.* ## Tech stack options ### Frontend framework React is the pragmatic choice specifically because the best canvas libraries target it. SvelteKit or Vue would work fine for the chat UI itself, but you'd be fighting the ecosystem on the canvas piece, which is the hardest part of this app. Next.js is reasonable if you want file-based routing and easy deployment; a plain Vite + React app is lighter if you'd rather skip Next's server/client split for what's mostly a client-heavy app. ### The canvas library — the decision that matters most - **React Flow** (now published as `@xyflow/react`) is purpose-built for exactly this: nodes and edges, pan/zoom/minimap, custom node renderers. Since the data model already *is* a node-and-edge graph, this maps almost 1:1 — a database row becomes a node prop, a parent pointer becomes an edge. Start here; it gets you most of the canvas for a fraction of the DIY effort. - **tldraw** is closer to what powers Heptabase — a general-purpose infinite whiteboard where you can drop in custom shapes and embed arbitrary content. More flexible, but you'd build the tree-layout logic yourself. Worth a look if you later want freeform annotations or images living alongside chat nodes. - **Konva.js / Fabric.js / PixiJS** are low-level canvas rendering libraries — you'd hand-build node dragging, edge routing, and hit detection. Only worth it if React Flow's opinions become a real constraint; not a sane starting point. ### Backend - **Node.js (Fastify or Hono)** keeps you in one language across the stack and has first-class support for streaming responses. - **Python (FastAPI)** is a good pick if you think you'll eventually want retrieval/search features (embeddings, semantic search across branches) — the ecosystem is stronger there. - **Rust (Axum)** is the most performant option and sits naturally alongside systems work in Zig — the mental model (explicit types, no runtime surprises) carries over directly, though it's more upfront ceremony for what's a fairly simple CRUD service. ### Database A parent-pointer tree doesn't need anything exotic: - **Postgres** for a normal hosted, multi-device app — recursive CTEs handle "give me the full ancestry of this node" in one query. - **SQLite** for a local-first build — single file, zero ops, pairs naturally with a desktop packaging tool. - Skip graph databases like Neo4j — they're built for queries you won't run (arbitrary multi-hop traversals across a dense graph). This graph is a tree; a plain relational table with a `parent_id` column is the right level of complexity. ### Local-first vs. hosted Worth deciding early since it changes several other choices at once. Chat history is sensitive-ish personal data, so a local-first build — **Tauri** (Rust-based, much lighter than Electron) wrapping the React frontend with SQLite on disk — keeps everything on the user's machine. The alternative is a conventional web app (Postgres + hosted backend) if the tree needs to be reachable from a phone or another machine without syncing files around. Starting local-first and adding sync later is easier than going the other direction. ### Streaming responses Anthropic's Messages API streams over Server-Sent Events, so mirroring that from your own backend to the client is the natural choice — it's one-directional, which is all you need. No reason to reach for WebSockets unless live multi-device sync gets added later. ## Should it support only Anthropic to start? Yes. Build it single-provider first — it removes a whole category of complexity (per-provider auth, differing message formats, different streaming event shapes) while the branching UX, the actually novel part of this project, is still being validated. The way to avoid painting into a corner: define a small interface up front — something like a `ChatProvider` with a `stream(messages, model)` method — and have the Anthropic integration be the first (and initially only) implementation of it. The backend and UI never talk to the Anthropic SDK directly; they talk to the interface. Adding OpenAI or Gemini later becomes writing a second class that satisfies the same interface, not a rewrite. This is the standard adapter/strategy pattern, and it's exactly what the "LLM adapter" box in the diagram above represents. ## Importing existing chats Both Claude.ai and ChatGPT let you export conversation history as JSON through their data-export features (worth checking the current export format when you get here, since these change). An importer just needs to walk that flat, linear message list and create one node per message, chaining each to the previous one as its parent — effectively importing it as a single unbranched thread. Once it's in the tree, the user can branch off any point in that imported history exactly like a native conversation. Treat this as a phase-3-or-later feature rather than something to build on day one — it's not needed to validate the core idea. ## Features that would raise this from useful to great - **Branch diffing** — a side-by-side view comparing how two branches diverged from their common ancestor, the direct git-diff analogy. - **Auto-titled nodes** — a cheap, small LLM call that summarizes a branch into a 3–4 word canvas label, so the tree is scannable instead of a wall of "Message 47" boxes. - **Full-text search** across the whole tree, not just the current branch — useful once trees get large. - **Manual context composition** — rather than a true git-style merge (which doesn't really make sense for prose), let the user hand-pick messages from two branches to seed a new branch's starting context. - **Export any branch path** as a clean markdown or docx transcript. - **Keyboard-driven canvas navigation** — vim-style keys (hjkl to pan, a number-prefixed jump between siblings) for moving around the tree without touching the mouse. - **Per-branch model picker**, once the provider abstraction exists — ask the same question as two sibling branches, one per model, to compare answers directly. ## Suggested build order The riskiest unknown isn't the canvas — it's whether the branch-and-navigate UX actually feels good in practice. This order gets that answered as cheaply as possible before investing in the expensive part (canvas rendering): 1. **Ship a linear chat MVP** — Build a single-thread chat UI that calls the Anthropic Messages API directly with streaming, no branching or canvas yet. This just proves the API integration, streaming, and message flow work end to end. 2. **Introduce branch-and-tree storage** — Extend the schema so every message row has a `parent_id`, add a "branch from here" action on any assistant reply, and build a plain list or tree sidebar (not canvas) to move between branches. This validates whether the branching UX itself feels good before investing in canvas rendering. 3. **Render the tree as an infinite canvas** — Swap the sidebar for React Flow, mapped directly onto the existing branch tree. Clicking a node focuses that branch in the chat panel; panning and zooming become the primary way to navigate larger trees. 4. **Add the provider adapter and chat import** — Wrap the Anthropic calls behind the `ChatProvider` interface so other models can be added later without a rewrite, and build an importer that turns an exported ChatGPT or Claude conversation into a starting branch. 5. **Layer in the extras** — Branch diffing, auto-generated node titles, full-tree search, markdown/docx export, and UI polish like keyboard navigation. > Phase 2 is deliberately the cheap, ugly version of branching (a list, not a canvas) so you find out fast whether the core interaction is worth building a whole canvas around, before sinking time into React Flow integration. If phase 2 feels wrong, it's much cheaper to rethink the data model at that point than after the canvas is built on top of it.