On the inscriptions of the Roman Republic, a medial dot separates the words. The practice goes back to the third century before our era and holds for a long time on monumental stone. In manuscripts it recedes: under the Empire, learned Latin sets its letters in the Greek manner, without interruption, and scriptio continua becomes the norm of literate writing for more than half a millennium.
The distribution is instructive. The separator holds where the reader is a passer-by, standing before a dedication they have not prepared. It disappears where the reader is a scholar who knows the language, the genre, often the text itself.
Rome did not lose a tool. It stopped needing one wherever someone could do the work in its place.
That work can be counted. In a chain of n signs there are n minus 1 positions between signs, each of them cut or not.
$$|S(n)| = 2^{n-1}$$
A line of forty letters therefore admits more than five hundred billion segmentations. The ancient reader keeps one. They eliminate the rest without thinking, with their language and their voice as sole instruments. Scriptio continua is not an impoverished writing: it is a writing that delegates the heaviest part of the operation to the reader.
The device reappears when the delegation becomes impossible.
In the seventh and eighth centuries, Irish scribes copy Latin. Latin is not their language. They lack the oral competence that let the Roman scholar absorb the flow, and they work on a corpus they cannot anticipate. They first break the text into blocks, then separate the words. The practice spreads through the British Isles, crosses to the Continent at the end of the tenth century with the reception of technical and philosophical texts, and becomes general in the thirteenth and fourteenth.
The blank appears exactly where the competence is missing.
The operation is not a graphic improvement. It is a transfer. The cut migrates from the reader's body to the surface of the page. What every reader redid at every passage, invisibly, becomes a fixed property of the object, performed once by the scribe and inherited by all the others. The word ceases to be an act of reading. It becomes a feature of writing.
The transfer has a price, and the reader pays it. As long as the cut remained their charge, it remained open: several segmentations were available, and the text could carry more than one. A separated manuscript has decided before them. The competing segmentations are not refuted, they are withdrawn. The reader receives a faster text and loses the right to cut it otherwise.
The gain, for its part, can still be measured. In Roman script, removing the blanks costs thirty to fifty percent of reading speed and shifts the fixation point from the centre of the word towards its beginning. The device does not make reading more pleasant: it changes its motor regime.
Then come the consequences nobody asked for.
In 1230, at the convent of Saint-Jacques in Paris, the Dominicans complete under Hugh of Saint-Cher the first verbal concordance of the Vulgate. It quotes nothing. It lists words and, for each, the places where it occurs. Verses do not yet exist: the address has to be manufactured, and each chapter is divided into seven sections marked by a letter. The concordance holds only if the word is isolable, countable, alphabetizable. Two centuries after the blank became general, the word has become an address.
From there, everything else. The dictionary presupposes the entry, the entry presupposes the cut. Word count becomes a measure of length, a contract clause, a threshold of permissible quotation, a unit of billing for those who write and those who translate.
We never count words. We count the cuts made by a device.
The contingency of that device is verified by comparison. Japanese is written without blanks. Inserting spaces into a text in pure hiragana facilitates word identification and eye guidance; in ordinary mixed script, kanji and hiragana together, the space brings nothing, the alternation of graphic systems already performing the function. Thai, which does not separate either, gains in speed when blanks it does not use are added to it. The consonantal scripts of the ancient Near East, for their part, kept a separator early: where the notation does not record the vowels, it compensates elsewhere.
The function is stable, its form is arbitrary. No writing escapes the obligation to supply a segmentation, by the blank, by the character, or by the voice of whoever reads.
The hard point remains. One might believe that the blank made visible a unit that was already there. It was not. Haspelmath has reviewed ten criteria used to define the word: possible pause, free occurrence, mobility, uninterruptibility, non-selectivity, and the rest. None is both necessary and sufficient. No combination yields a definition that coincides with the orthographic practice of linguists themselves. The discipline cuts as the printer cuts, then theorises on that cut.
Linguistics did not find the word. It inherited it from a scribe who did not speak Latin.
Doctrine
A flow has no units. Units are produced by a segmenting device, and that device sits in one of two places: in the competence of the one who receives, or in the matter of what is transmitted. The second location is not a copy of the first. It stabilises the cut, makes it transportable, and removes it from discussion.
An inscribed cut fixes the unit of account of everything that will follow: what can be indexed, cited, counted, billed, owned. The order is irreversible. Counting inherits the cut and never examines it.
The loss is double. The discarded segmentations cease to be available, and the one who receives loses the competence that made them available. A device that takes over an operation ends by abolishing the capacity to perform it.
Vecteur ouvert
A language model does not read words. It reads tokens, produced by a subword segmentation learned from a corpus, obtained in 2016 by adapting a compression algorithm published in 1994. The cut there is optimised for frequency, not for meaning. It coincides with no linguistic unit and varies from one model to the next.
The Irish scribes cut badly, but they cut aiming at the word. Here nobody claims the cut means anything. And it is nonetheless the cut that fixes the unit of account: context length, the price of inference, the granularity of what a system can learn of a language are all measured in tokens.
Separated writing took six centuries to impose its unit on lexicography, on law and on publishing. The question is therefore not whether tokenization is faithful: it is not, and does not claim to be. When the unit of account of a system no longer has any counterpart in the language it processes, what still guarantees that what it cuts and what we read are the same object?
