General Text & Computing Company

Real-time Collaboration

This is the third page of the guide to building General Text apps. It covers the deep collaborative-editing path: merging by structure rather than characters, live cursors and presence, and the bundler setup that makes real editor bindings work. It builds on the file API in The window.gt Runtime, so read that first.

Reach for any of this only when the character-merge of your serialized format genuinely corrupts under real concurrent editing. If your docs are effectively single-user, the high-level string API from the runtime page is the right tool and none of this applies.

Rich structured sync: the per-file Y.Doc

Rich structured sync, the deep escape hatch. A Y.Text merges characters. For structured content (a ProseMirror document, a table, an outline) you want a CRDT that merges structure, so two people reordering list items or editing the same table cell converge correctly instead of producing a mangled string. gt.subscribeFileDoc(path) hands you the live per-file Y.Doc so you can put any Yjs type on it (e.g. a Y.XmlFragment for y-prosemirror, a Y.Array for rows) and bind your editor to it.

One rule, and it's the whole contract:

  • content is canonical. The content Y.Text on this doc, exactly what gt.subscribeFile(path) returns, is the only key written to the stored file. That keeps the artifact a real, portable .md/.csv/.json the user owns and any other app can read.
  • Your structure is a sync channel, not storage. Anything else you put on the doc (getXmlFragment('pm'), getArray('rows'), …) syncs across clients so your merge is correct, but is never persisted to disk. You are responsible for keeping content updated as the plaintext projection of your structure.
  • You must handle out-of-band edits. Because content is canonical, something else can change it: a filesystem edit on desktop, another app, a git pull. The platform tells you when this happens via gt.on('file-changed-externally', path), a signal that fires ONLY for a real external edit, never for your own or a peer's projection. On it (or on next open, for sole-writer apps), rebuild your structure from the text (re-parse it). How lossy that is depends on your format: CSV/JSON round-trips cleanly; rich text loses structure markdown can't express. That's the cost of keeping the file plaintext, so design your projection to minimize it.
  • No per-key privacy. There is no access control within a file's doc: anyone with read access to the file sees every key on it, your structure channel included. That's fine (read access to the file already means read access to its content), but don't treat the structure channel as a place to hide anything you wouldn't put in the file itself.

subscribeFileDoc is fully general: it hands you a plain Y.Doc. y-prosemirror is just one example; use any Yjs type (a Y.Array of Y.Maps for a spreadsheet, nested maps for an outline, …). The platform is editor-agnostic: it never inspects your structure, only ever reads the content key.

const path = 'notes/today.md'
const fileDoc = gt.subscribeFileDoc(path) // one per-file Y.Doc…
const fragment = fileDoc.getXmlFragment('pm') // …the sync-only structure channel
const content = gt.subscribeFile(path) // …and its CANONICAL text (same doc)
const serialize = () => serializeToText(view.state.doc) // your schema → text

// Bind y-prosemirror: ySyncPlugin(fragment) + yUndoPlugin(...) — and DROP
// prosemirror-history (its undo would revert collaborators' edits; see Pitfalls).

// SEED only a brand-new file — but WAIT for the server's state first, or you'll
// seed a file whose content simply hasn't arrived yet (→ clobber/duplicate it).
// (Opening the same new doc in two windows? Elect one seeder — see Pitfalls.)
await gt.whenFileSynced(path)
let ready = false
if (gt.files().includes(path)) {
  buildStructureFrom(content.toString()) // server's content is now reflected
} else {
  buildStructureFrom(STARTER_TEXT)
  gt.applyDiff(content, content.toString(), STARTER_TEXT) // write the new file
}
ready = true // only now may we project (guards the empty-doc wipe — see Pitfalls)

// Project structure → canonical text. GATED on `ready`, and never write text that
// already equals the canonical content (so an empty/transient doc can't wipe it).
fragment.observeDeep(() => {
  if (!ready) return
  const text = serialize()
  if (text === content.toString()) return
  gt.applyDiff(content, content.toString(), text)
})

// NOTE: there is deliberately NO live content→structure rebuild here. If your app is
// the SOLE writer of this file (the common case), you don't need one — structural
// edits already sync between clients via the fragment, and an external edit
// (filesystem/git) is picked up the next time the file is opened (the seed step above).
// A live `content.observe → buildStructureFrom` is the #1 source of duplication/cursor
// bugs: two clients can rebuild near-simultaneously and their clear+reinsert merges
// into a doubled document. Don't watch `content` to do this — instead, when you DO want
// to refresh mid-session on a genuine external edit, listen for the platform's signal,
// which fires ONLY on a real out-of-band edit (never on your own or a peer's projection):
gt.on('file-changed-externally', (changed) => {
  if (changed !== path) return
  // Safe: this is a real external edit, so rebuild (or prompt the user to reload).
  buildStructureFrom(content.toString())
})

// Teardown: ONE unsubscribe releases the doc + structure + awareness for this path
// (there is no unsubscribeFileDoc/unsubscribeFileAwareness — see Pitfalls).
// awareness.setLocalState(null); gt.unsubscribeFile(path)

Pitfalls (these are silent, so read them). The projection loop is the genuinely hard part; each of these has bitten people:

  • Be the sole writer if you can. The simplest, safest design: your app is the only thing that writes this file. Then you project structure → content, build structure from content only at open (the seed step), and never wire a live content → structure rebuild. Reach for a live rebuild only if a non-app writer (filesystem, git, a different app) genuinely edits the file mid-session.
  • Empty doc wipes the file. An editor mounts with an empty doc and fires a transaction before you seed. If projection runs then, it writes '' over real content. Gate projection until the structure is seeded (the ready flag) and never project text that already equals the canonical content.
  • Refresh on external edits with the signal, not by watching content. content.observe can't tell a real external edit from a peer's projection echo, so a content.observe → buildStructureFrom rebuild is dangerous: two clients rebuilding near-simultaneously merge their clear+reinsert into a duplicated doc + cursor jump. The platform disambiguates for you: gt.on('file-changed-externally', path) fires ONLY for a genuine out-of-band edit (a filesystem/git/other-tool write), never for your own or a peer's projection. It reaches every client with the file open, wherever the edit happened (a desktop disk edit propagates to web collaborators too). Rebuild (or prompt the user to reload) from that, not from a raw content watch. If you must hand-roll a content watch anyway, guard hard: skip while the editor is focused (view.hasFocus()), debounce, compare text === serialize() (so a peer/your-own echo is a no-op, never "did I write last?"), and re-verify after the debounce.
  • Make your text↔structure round-trip idempotent. A rebuild (on the file-changed-externally signal, or at open) re-runs your projection, so serialize(parse(text)) should equal text. If the round-trip normalizes the text (*x***x**, whitespace, reordered attributes), your app rewrites the file on every external edit, churning it for collaborators and risking oscillation when two clients normalize differently. The text === content.toString() projection guard only suppresses an exact echo; a normalizing round-trip slips past it.
  • Cold-open seeder election (multi-window hits this every time). Two windows on the same brand-new doc both pass whenFileSynced (locally it only waits for IndexedDB, not the other tab) and both seed → duplicated content. Elect ONE seeder via awareness: after await gt.whenFileSynced(path) and a short grace (~250 ms for presence to populate), seed only if the fragment is still empty and you hold the lowest clientID present; everyone else adopts the synced structure.
    await gt.whenFileSynced(path)
    await delay(250) // let awareness from other windows arrive
    const aw = gt.subscribeFileAwareness(path)
    const myId = fileDoc.clientID
    const lowest = Math.min(myId, ...[...aw.getStates().keys()])
    if (fragment.length === 0 && !gt.files().includes(path) && myId === lowest) seed()
    
  • One doc, one unsubscribe. subscribeFile / subscribeFileDoc / subscribeFileAwareness for a path return views of the same per-file doc; release it all with gt.unsubscribeFile(path). There is no per-variant unsubscribe. Calling awareness.setLocalState(null) on unmount is polite (disconnect auto-drops presence anyway).

Most apps never need any of this. Reach for the structure channel only when character-merge of your serialized format genuinely corrupts under real concurrent editing. If your docs are effectively single-user, the high-level string API is the right tool.

One Yjs instance (read this before you import 'yjs')

This is the single thing that trips up apps using y-prosemirror, y-codemirror.next, or any Yjs binding. Yjs relies on instanceof, and those libraries import 'yjs' themselves, so if your bundle ships its own copy of Yjs, you now have two Yjs instances: the one the runtime used to create the Y.Doc/Y.Text/Y.XmlFragment you got from subscribeFile*, and the one your editor binding uses. The moment a runtime-created object meets your binding, every instanceof check fails, usually as a baffling "invalid content" / "expected YXmlFragment" error, not a clean one.

The fix: share the one instance the platform already serves. The runtime publishes two tiny facade modules, /__gt/yjs.js and /__gt/y-protocols-awareness.js, that re-export window.gt.Y / window.gt.awarenessProtocol under every name the real package exports. Mark yjs and y-protocols/awareness as external, resolved to those URLs, so they load at runtime instead of being bundled. Do it with a tiny resolveId plugin, not resolve.alias: an alias makes Vite try to resolve /__gt/yjs.js as a local file in dev (it isn't one) and you get the very Failed to resolve import "yjs" this section exists to prevent; build.rollupOptions.external is build-only and doesn't help dev. A resolveId that returns external: true is the one thing that works in both dev and build:

// vite.config.js
function gtSharedYjs() {
  // Leave these imports as `/__gt/*` for the browser to load at runtime (dev AND build).
  const ext = {
    yjs: '/__gt/yjs.js',
    'y-protocols/awareness': '/__gt/y-protocols-awareness.js',
  }
  return {
    name: 'gt-shared-yjs',
    enforce: 'pre', // run before vite's own resolver
    resolveId: (id) => (ext[id] ? { id: ext[id], external: true } : null),
  }
}

export default {
  plugins: [gtSharedYjs()],
  // CRITICAL: exclude every binding that itself `import 'yjs'` (y-prosemirror,
  // y-codemirror.next, …) from dep pre-bundling. Otherwise esbuild pre-bundles the
  // binding and resolves ITS `import 'yjs'` to the installed package — bundling a
  // SECOND Yjs, the exact dual-instance bug above. (The resolveId plugin doesn't run
  // inside esbuild's pre-bundle pass, so excluding the binding is what forces its
  // `yjs` import back through the plugin.)
  optimizeDeps: { exclude: ['y-prosemirror'] },
  // In `vite dev`, `/__gt/*` isn't a file on your dev server — proxy it to the platform
  // so the external imports load at runtime. (In production the app frame serves these
  // same-origin; this is dev-loop only.)
  server: { proxy: { '/__gt': { target: 'https://www.generaltext.org', changeOrigin: true } } },
}

Now your code, y-prosemirror, and the runtime all resolve to the single instance the platform serves, so instanceof holds, objects cross freely, and Yjs drops out of your bundle. Because the imports are external, your bundler never looks inside them, so there's no export name-list to maintain and the 'X' is not exported build error can't happen: the platform's facade re-exports the complete API, matched to the runtime's version. (Marking them external is what makes this reliable. Without it the bundler tries to statically resolve the names at build time, which is the trap that only fails at deploy.)

A few things that trip people up here:

  • Share only yjs and y-protocols/awareness. Everything else (lib0, all prosemirror-*) bundles normally. lib0 is stateless utilities; ProseMirror objects aren't the runtime's. Don't alias them or you'll create new problems.
  • Keep yjs and y-protocols installed as devDependencies. Marking them external removes them from the bundle, but TypeScript still resolves y-prosemirror's .d.ts against them, and "don't bundle Yjs" is not "don't install it."
  • A stray dev Pre-transform error: Failed to load url /__gt/yjs.js is non-fatal. Vite eagerly pre-transforms the external target; if the proxy can't reach the platform at that instant it logs this once. The browser still fetches /__gt/yjs.js through the proxy at runtime, so the app loads. (If it persists, your server.proxy target is wrong or the platform is unreachable.)
  • The facade reads window.gt when it loads. /__gt/yjs.js reads window.gt.Y at evaluation, so the runtime must already be present. Inside General Text it's injected before your modules, so it's set. But if you self-host a demo/landing page that injects the runtime itself, a static import of your editor pulls in the facade before your <script src=runtime.js> runs → crash. Load the editor lazily after the runtime exists: await runtimeReady; const { mountEditor } = await import('./editor').

If you'd rather not touch your bundler at all, the alternative is to never let runtime objects meet your binding: keep your own bundled Y.Doc, bind the editor to that, and bridge updates as bytes: gt.subscribeFileDoc(path).on('update', u => myYInstance.applyUpdate(myDoc, u)) and the reverse via gt.Y.applyUpdate(runtimeDoc, u). Only Uint8Array updates cross the boundary, so the two instances never compare objects. More code, zero build/alias/types config.

Presence & cursors

gt.subscribeFileAwareness(path) gives you a y-protocols Awareness bound to a file's doc, synced to every other client viewing the same file. It's the channel for live cursors, selections, "who's here" avatars, and typing indicators. Like the structure channel it's ephemeral: never written to the file, and a client's presence is dropped automatically when it disconnects.

Two ways to use it. Custom presence, set and read fields directly:

const awareness = gt.subscribeFileAwareness('notes/today.md')

// Publish your presence (anything JSON-serializable):
awareness.setLocalStateField('user', { name: 'Ada', color: '#e91e63' })

// React to everyone's presence (yours + peers'):
awareness.on('change', () => {
  const here = [...awareness.getStates().values()] // [{ user, cursor, … }, …]
  renderAvatars(here)
})

Or hand it straight to y-prosemirror's cursor plugin (alias y-protocols/awareness to gt.awarenessProtocol first; see "One Yjs instance"):

import { ySyncPlugin, yCursorPlugin, yUndoPlugin } from 'y-prosemirror'

const fragment = gt.subscribeFileDoc('notes/today.md').getXmlFragment('pm')
const awareness = gt.subscribeFileAwareness('notes/today.md')
// editor plugins: [ ySyncPlugin(fragment), yCursorPlugin(awareness), yUndoPlugin(), … ]
// Use yUndoPlugin and DROP prosemirror-history — plain history undoes by document
// position and will revert your collaborators' edits, not just yours.

subscribeFile, subscribeFileDoc, and subscribeFileAwareness for a path all ride the same per-file doc, so structure, content, and cursors stay coherent. Presence works on any file you can read; you don't need write access to show that you're viewing.

Presence fields are app-asserted, not identity-verified. Whatever user/name/color you put in an awareness state is relayed as-is to other clients on that file; the platform does not stamp or verify it. That's fine for cursors and "who's editing" among collaborators who already share the file, but don't use an awareness field as proof of identity for any security or trust decision.

y-prosemirror wiring gotchas

  • dispatchTransaction and the new EditorView TDZ crash. ySyncPlugin dispatches a transaction synchronously inside new EditorView(...) to seed the doc from the fragment, before your const view = new EditorView(...) has been assigned. A custom dispatchTransaction that closes over view throws can't access 'view' before initialization, and only when the fragment is non-empty (an empty-doc smoke test misses it). Fix: reference the view via this inside dispatchTransaction (ProseMirror binds it), not the const you're assigning.
  • Never do per-transaction work. y-prosemirror emits several transactions per keystroke (sync, cursor, relative-position). If your dispatchTransaction walks the doc or calls setState every time, typing crawls (seconds per keystroke is easy to hit). Coalesce selection/toolbar updates to one per animation frame and skip when unchanged; push whole-doc work (review panels, comment re-anchoring) off the keystroke path (debounced); never setState synchronously per transaction.
  • Cursor color must be hex. yCursorPlugin does alpha math on the awareness color and throws "unsupported color format" on hsl()/named colors. Use #rrggbb (e.g. #e91e63).