A token is a small piece of text that a language model reads and writes. AI in the cloud is often paid per million tokens, so the number of tokens drives the price.
A token is a small piece of text. A language model splits everything into tokens before it reads it. A token can be a whole word, part of a long word or a character such as a full stop.
Why do tokens matter?
Price: AI in the cloud is often paid per million tokens. That applies both to what you send and to what you get back. An agent that tries several times can use many tokens. That is why the price is hard to predict. Look instead at what it costs to solve one task.
Space: A model can only remember a certain number of tokens at a time. This is called the context window.
Tokens and Domino IQ
If Domino IQ runs the model on your own server, you do not pay per token. You pay for hardware and power. That can be far cheaper if you use AI a lot.
