altAI
An open-weight model split across several computers, behind one OpenAI-compatible endpoint. A machine that cannot fit a model scores zero, not slow. Several of them together can serve it.
Compute you can wait for. Capacity you either have or you do not.
Download
The engine is inside. No terminal, no Python, no Docker, no address to type — the network's address is already in the app. You will need a join token; the network is invite-only while it is this young.
xattr -dr com.apple.quarantine "/Applications/Altai Node.app"
export ALTAI_JOIN_TOKEN=<your token>
open "Altai Node.app"Unsigned, so macOS asks once — that is the xattr line.
Both builds run on macOS 12 and later, measured rather than claimed:
every library in each archive is built against that floor. The Intel engine gives up
Accelerate’s BLAS to get there, which makes prompt processing slower on an Intel
Mac — there is no Metal on those to take over.
Windows: unzip, double-click
altai-node.exe, and choose More info → Run anyway — it
is not code-signed yet. The engine is a CPU build of llama.cpp and needs the Visual C++
Redistributable, which most machines already have; the folder’s read-me says what
to do if yours does not. Linux: the node runs and joins, but the
archive carries no serving engine — no published llama.cpp build
has the RPC backend splitting needs, so it must be built from source and placed beside
the binary. The README in the archive says how. Both come from one CI run, built
1 Nov 2026.
What a token gets you
An ordinary token can only ever make your machine a stage: it runs a block of layers and nothing else. It never downloads a model, it is reached only by its own group's head, and nothing on the internet can dial into it. Being trusted to drive a group is a separate privilege the token has to carry, and we do not hand that out.
Architecture
Three roles, and any machine can hold any of them. Which one you are offered is decided by what you choose to contribute — the coordinator fits the model to the memory you declared. It never holds model weights and never sees a group’s internal traffic.
Lowest bar. Holds a block of middle layers and runs them for every token. Never downloads a model file and never holds a usable copy of one — its head pushes the tensors it needs. Sees hidden states, never a prompt.
Drives a group. Holds the embedding and output layers plus its own block, emits every token, and signs for what it served. Needs the model file on disk. Because it reads prompts in plaintext, this one is a privilege rather than a size — see below.
Highest bar, best performance. Holds a complete model and serves a request start to finish, with no hop between machines and so nothing of the per-token round trip a group pays.
Why head is not simply the next size up
A head receives each prompt and completion in plaintext and signs
the receipt for it. Capacity decides what a machine can be; the invite code
decides what it is allowed to be. A machine large enough to head a model may
still be offered only stage, and the page will say so rather than leave
you guessing. A stage needs no such trust, which is why it is the role anyone can take
today.
The argument
The binding constraint on a frontier-class model is memory, not FLOPs. A fast GPU with 24 GB cannot hold a 600 GB model at any clock speed. Twenty of them can.
And capacity is what is scarce. Datacenter servers are supply-constrained — vendors say so in their own filings and expect it to persist — while the memory in workstations, gaming machines and office hardware is already built, already powered, and mostly idle. It is not waiting on a fab.
So the aim is not to beat a hosted API on price or latency. It is to serve a model that otherwise could not be served at all, for the people who need it, on machines that already exist.
The target
Sparse mixture-of-experts models — DeepSeek-V3, Kimi-K2 — are the problem this exists to solve. They are where the memory wall is worst and where it is most tractable: only a fraction of the experts fire for any token, so the weights that must be resident are far smaller than the weights that must be available.
The route is pipeline stages across a low-latency regional group, each stage running its own block of layers with that block's experts offloaded to ordinary system RAM. Experts never cross the network. Only hidden states do — on the order of 16 KB per token per hop. That is what makes a machine with plenty of RAM and a modest GPU useful, instead of only machines with datacenter cards.
This route does not run yet
In the engine we use, expert offload and pipeline parallelism are currently mutually exclusive — switching on offload disables the pipeline, verified in the engine's own source. The upstream change that would lift it is open and unmerged. Until then the sparse frontier is the design target and not a capability, and nothing here should be read as saying otherwise.
State
Roadmap
Intentions, not dates. Each one is listed because something concrete blocks it, and the blocker is named.
The shape we are building toward
Ten machines of 64 GB, in one metro, serving a DeepSeek-V3-class model at roughly 10–20 tok/s. A target, not a result — nothing like it has been run.
Two numbers in that sentence are not taste. Per-token time for one person waiting is
W/B + crossings × (c + L) — and the first term
does not divide by the number of machines. A stage reads its share, but
every stage runs in sequence to make one token, so the same bytes get read either way.
Splitting relocates the work; it does not shrink it. There is no optimum group
size, only a floor: the smallest group whose memory holds the model. Every
machine past that adds capacity and latency, never speed.
Which leaves the crossings, and those are the whole game. Each boundary costs about a millisecond of protocol — measured — plus the network round trip, and the network is the larger number by far. So distance is the lever, not machine count. Ten machines a metro apart land in the target range; the same ten spread across a continent do not. And availability compounds: ten nodes at 99% give 90%, forty at 95% give under 15%.
So it is ten large machines close together, not forty small ones far apart. More-and-smaller is the intuitive answer and the arithmetic says it is wrong. Read “64 GB” as a floor on capacity — what sets speed is memory bandwidth, where two machines of the same size can differ sixfold.
Corrected 4 Nov 2026. This box previously derived an
optimum group size from W/(N·B) + N·L — sound algebra on
the wrong quantity, since that form describes aggregate throughput under load, not one
person waiting. Our own measurements had contradicted it for a week before the algebra
was checked. The figures above are arithmetic from measured inputs, not results: nothing
of this shape has been run.
Limits
It does not serve frontier models today. What has actually run is a 14B open-weight model split across two Macs; the near-term target is 32B–70B dense. The sparse-frontier ambition above is the problem being worked on, not a description of what you would get this morning.
It is not cheaper than a hosted API. At the speeds a pipeline across several machines runs, electricity alone costs more per token than the market price of a hosted model. The argument is capability, not price.
A receipt will be an attestation, not a proof. When they exist, a receipt will say that a node asserted it produced an output. A dishonest node can sign a false claim, and nothing here will imply otherwise.
Operators
One file, opened. It profiles the machine, proposes how much to contribute, opens a page in your browser, and joins. No terminal, no Python, no Docker, no address to type. You set the ceilings; the coordinator never asks for more than you allowed.
export ALTAI_JOIN_TOKEN=<the token you were given>
open "Altai Node.app"A stage downloads nothing — its head sends it the tensors for its own layers. Only a
head or a whole node needs the model file on disk.