Sign up to keep track of your learning progress.
Log inHow is AI made?
You don't need to know code. Feed in data and watch AI being built through tokenization → vectorization → training → generation. If you want, you can also build it with real Python code.
A toy word-bigram model: it looks only at the previous word. It does not learn embeddings, attention, or facts. The word splitter is a teaching example, not an LLM tokenizer.
- Data prep
- ① Tokenization
- ② IDs and one-hot vectors
- ③ Training
- ④ Generation
Data prep
Collect a few short example sentences. Repetition changes the counts; more text is useful only if it fits your purpose. Each sentence ends separately.
① Tokenization
This demo splits sentences into word-like pieces. Real LLM tokenizers can use words, word pieces, characters, and bytes. Tokenization is about representing text, not about text being too long to understand.
② IDs and one-hot vectors
An ID is a number used as a label. A one-hot vector marks that ID's position with 1. Neither tells us which words have similar meanings. A learned embedding is a different kind of vector; this demo does not train one.
③ Training
Count each transition, including sentence endings. A candidate's probability is its count divided by the total count after the selected word. Neural language models learn weights instead of just storing this small table.
④ Generation
Choose the most likely candidate (greedy), or draw according to probabilities (sampling). Sentence-end stops generation. Sampling can produce the same result again. Generated text is not checked for truth.
| Word | Number | Number vector (one-hot) |
|---|---|---|
Pick a word to see what the AI learned about which word is likely to come next:
Percentages are rounded; their displayed total may differ slightly from 100%.
This word only appears at the end of a sentence, so there's no next word. Try another word.
The table shows observed frequencies at temperature 1. Temperature changes sampling probabilities, not the learned counts or factual accuracy.
Preparing the Python runtime…
My Code
Output
Press [Run] and the result will appear here.