Firebrat has two independent halves that only touch through a portable book-package format: a conversion pipeline that runs once per book, and a Flutter reader that never needs the pipeline again afterward. A second, optional pipeline for multi-voice novels ends in the same package format.
Layout-aware parsing (marker-pdf + PyMuPDF) pulls per-page text, headings, figure crops, and full-page renders out of the source PDF, assigning every figure/table a stable id.
An LLM groups raw pages into logical sections, rewrites dense text into clean narration, and — critically — recognizes mathematical relationships in the prose and authors real LaTeX for them.
A local text-to-speech model synthesizes one audio clip per sentence, then Firebrat concatenates them and tracks each clip's real, sample-derived duration to build exact start/end timestamps.
Manifest, audio, and assets are bundled into one self-contained .tar.gz — servable over HTTP, or copied straight onto a device.
The naive assumption going in was that a PDF's formulas could be extracted as LaTeX the same way its figures are extracted as images. That assumption held for exactly zero real formulas in testing. Real technical books very often typeset math as plain inline text — tCK = 1/f = 10 ns — not as a distinct, taggable equation block. There's nothing for a PDF parser to extract, because there's no structured math in the source file to begin with.
So formulas in Firebrat are LLM-authored, not extracted. The same model that writes narration text is asked to recognize a mathematical relationship in the surrounding prose and write real LaTeX for it directly. Firebrat's own code — never the model — then mints the formula's id and adds it to the book's asset catalog. This keeps the same invariant that governs figures and tables (an id always comes from code, never from a model's output, which removes an entire class of hallucination bugs) while acknowledging that the LaTeX itself has to come from somewhere the extraction stage simply doesn't have access to.
An alternative design would synthesize a whole section's narration as one long audio clip, then run a separate forced-alignment model afterward to figure out which words correspond to which timestamps. Firebrat does the opposite: each sentence is its own short TTS call, and the timestamps fall out for free from literally measuring how long each resulting clip is and adding a fixed silence gap between them. No alignment model, no alignment failure modes, no approximate boundaries — the highlight sync is exact by construction, not by inference.
A mobile audiobook reader should work on a train with no signal. The backend's entire job is producing a complete, self-contained package and handing it over — once — after which the app never needs to talk to it again. That's also what makes a book a genuinely portable file: the same .tar.gz the backend serves over HTTP can just as easily be copied to a phone over USB, shared through a messaging app, or dropped in cloud storage, and the app's built-in file picker will open it identically either way.
A second pipeline, for novels, lives beside the technical-book one. It turns an ebook into an audiobook with a distinct voice per character, and ends in the same package format, so no client needs to change.
Gutenberg EPUB or an uploaded ebook (or a text PDF) becomes clean text, one raw page per unit, boilerplate removed for Gutenberg books.
A deterministic split into quoted speech and narration, each with a stable page and index key.
The LLM builds the character roster, attributes every quote to a speaker and tags emotion. Resumable per chunk; output is cast.json.
Qwen3-TTS makes an anchor voice per character and speaks each line. Text is normalised, silence trimmed and loudness matched.
A standard Firebrat package, zipped with audio stored, ready for a client or the Internet Archive.
The cast stage reads the book in two passes: a roster pass over windows of about 3,500 words, then an attribution pass over chunks of at most 600 words and 30 quotes. Each chunk result is cached on disk, several chunks are sent to the model at once, and a regular-expression fallback keeps one bad reply from stopping a book. Stages run as separate processes, each in its own Python environment, so the GPU memory is fully released between them.
A 16 GB graphics card cannot hold the casting model and the speech model together, so the scheduler works in phases and batches books to avoid reloading models.
Each book is a small JSON file dropped in the scheduler’s inbox/ folder, either by a command-line submit or by the server when someone uploads with Expressive voices.
Downloads and text extraction need no GPU, so they run as soon as a book arrives.
The scheduler starts a managed vLLM server, casts all waiting books together (up to a configurable cap), then stops vLLM and checks the GPU memory is really free before going on.
Qwen3-TTS narrates the cast books one at a time, shortest first by default, and zips each. Books that arrive during audio wait for the next round. The LLM phase takes priority over audio.
Casting sends many small requests, so throughput matters. On an RTX 4080 (16 GB), vLLM serving Qwen3.5-9B with FP8 weights (8,192-token context, eight parallel sequences) measured about 152 tokens per second in aggregate, against about 2.3 tokens per second for the same model loaded in-process with 8-bit quantization. In practice casting ran at roughly 8,000 to 23,000 words per minute: a 26.7k-word book in 3 minutes 17 seconds, and an 84k-word book in 3 minutes 37 seconds.
Status. The pipeline code is covered by offline tests, and the casting stage has run on real novels on the GPU server. The speech stage has not yet been verified end to end on real books, and the scheduler itself has not yet been run as the long-lived service. Expect to tune voices after the first listen.
If you run your own Firebrat server you can turn on accounts, roles and remote control of the conversion queue. Everything here is off unless you enable it, so existing setups keep working unchanged.
| Environment variable | Purpose |
|---|---|
FIREBRAT_IAM=1 | Turn on accounts, roles and the admin routes. |
FIREBRAT_ADMIN_EMAILS | Optional comma-separated verified emails that become admin on sign-in. |
FIREBRAT_ALLOWED_USER_IDS | Older allow-list; with accounts on, its entries become admins while none exists. |
FIREBRAT_CAST_ENABLED=1 | Enable Expressive voices and let the server start and supervise the scheduler. |
FIREBRAT_CAST_ROOT | Working folder for inbox, per-book state and output. |
FIREBRAT_CAST_LLM_PY, FIREBRAT_CAST_TTS_PY | Python interpreters for the casting and speech environments. |
FIREBRAT_CAST_VLLM_BIN | The vLLM program the scheduler launches for the LLM phase. |
FIREBRAT_CAST_BATCH | Casting requests per batch (default 8). |
FIREBRAT_SETTINGS_FILE | Set by the server for the scheduler; points at the saved settings file. |
Status. Implemented and unit-tested in the server, the web app and the Android app, and reviewed for security. It has not yet been run against a live deployment: sign-in flows, the setup code, the scheduler reload and the admin screens have not been exercised end to end. One setting, the maximum characters for an Expressive upload, is stored but not enforced yet. Books and jobs made before accounts were turned on become admin-only once they are.
The repository's docs/ folder is the authoritative, continuously-updated reference — this page is the tour, those are the details.