graft

@alinea/graft · storage engine for alinea 3.0

A content store that speaks git and never conflicts.

graft keeps a site’s content in a git repository. Every copy can be edited at any time, online or not. When copies meet, their edits merge into the same content, and nobody is asked to resolve a conflict. Everything below runs the real library in this page; where a figure simulates the network, its caption says so.

  1. 01Two people, one page
  2. 02Shapes
  3. 03Local commits
  4. 04Any git host
  5. 05Editing together
  6. 06Dozens of copies
  7. 07Lenses

01

Two people edit the same page

Ana and Ben start from the same version of a page and change it at the same time. Edit either side and the merge runs again on every keystroke. The rule beside a field says whose change it kept, with both colours where it combined the two.

pages/home.json three versions, one merge

02

A shape tells graft what the content means

graft knows no field names. The shape says that a body is text, merged by line and then by character within the lines both changed, that tags are a set, and that blocks are a list whose items are matched by _id and ordered by _index. Switch a field to atomic() and it merges as one value: when both change it, one of them loses.

shape.ts drives the merge above

03

Saves are git commits, made locally

Every save is a commit in a repository the application owns, landed by a compare-and-swap of the branch. An editor saves against the commit they loaded. When someone else saved first, graft merges the save onto the new head and writes it on top, so history stays a single line.

refs/heads/main click a commit to read it
save(repo, ref, {base, changes, …}, {prefer})

04

Any number of git hosts, any of them down

The repository replicates to git hosts over smart HTTP, each on its own. A pass brings in what developers pushed and pushes the rest, never forced. A host that is down or refuses the push waits, and the others carry on.

Each commit graft pushes names the canonical commit it holds in a Graft-Content trailer, and each commit it brings in names the host and commit it came from in Graft-Source.

replicate three git servers in this page
Each host is graft’s own gitServer over a store of its own, reached by gitRemote({url, fetch}) through a fetch that throws while the host is down. A pass runs when you press the button, for all three at once.

05

Editing together, with no server holding state

For live editing, each person holds a CRDT of the page and syncs with a session log, which only orders and stores operations. A sync can wait on the server until there is something new, so others’ typing arrives within a round trip. Take yourself offline and keep typing: what you type waits and goes out when you are back. When the last person leaves, the session becomes one commit that credits everyone.

session you, and two people typing slowly
open, join, sync and leave over a MemorySessionLog, with a Participant each. The network is simulated: the three sync in turn, one every 0.3 seconds, where over HTTP each would hold a request open until there is news.

06

Dozens of copies at once

Here it all runs together: the canonical repository in the middle, three git hosts and a live session around it, and editors on the outside. A copy’s colour stands for the content tree it holds: copies that agree share a colour, and different trees get different colours, up to twelve at a time.

Take copies offline, cut links, kill a host. Then heal everything and watch every copy end on one tree.

swarm click a copy to take it offline, a link to cut it
The network is simulated: a seeded random source and a virtual clock decide who talks to whom, and when. Every commit, merge and sha is real, so a seed gives the same shas each time. The live session closes into main every hundred entries and a new one opens, as one does when its people leave.

07

Give someone part of it

A lens serves part of a repository as a git repository of its own. This one lets a translator read the locales without the keys that start with _, and write only the Dutch file. A push to nl.json becomes a commit on the real repository, by whoever signed in, with the hidden keys kept. A push that touches anything else is refused.

lens nl a repository inside a repository
new Lens({repo, name: 'nl', read: transformed(paths(['locales/**']), ['locales/*.json'], keys(codec, key => !key.startsWith('_'))), write: paths(['locales/nl.json'])}) from @alinea/graft/lens, and lens.push(from, to, {sign}) for each push.