Put the you trained yourself on so anyone can download it. Export with export.py, write a , then push.
is a site for uploading and downloading , a bit like for models. One model is one . Upload the file, settings, , the that runs it and a () together, and other people can download the same model and run it as is.
There are three steps. export.py writes the numbers in your checkpoint to a file, you write the model card, and git pushes everything to your Hugging Face repository. The does not change. Whoever downloads the model rebuilds the checkpoint with export.py --restore and runs it with the same generate.py, runner.py and xchat.py.
Hugging Face is a public hub for uploading and downloading models, and demos. Everyone from big companies to students uploads models there, and SmolLM2-135M from the overview is downloaded from there too.
When you publish, other people can run your result again exactly as it is. This is called reproduction. In real AI development, people download public models, fine-tune them and upload them again every day.
safetensors is a weight file format made by Hugging Face. It has only three parts.
The .bin files that were common before are pickle files, so code hidden inside could run the moment you opened one. safetensors holds only numbers and JSON, so opening it never runs code. That makes it safer when you download a model someone else uploaded.
's Python numbers are 8 bytes (float64), but the file stores them in 4 bytes (float32). A tiny difference appears, like 0.1 becoming 0.10000000149011612, but a model trained on the default generates exactly the same text. 1,248 numbers × 4 bytes = 4,992 bytes, plus the header, makes a file of about 5.7 KB.
import struct
from pathlib import Path
from export import write_safetensors, read_safetensors
path = Path('lab/tiny.safetensors')
write_safetensors(path, {'w': [[0.1, 0.5], [-2.0, 3.0]]})
raw = path.read_bytes()
size = struct.unpack('<Q', raw[:8])[0]
print('file bytes:', len(raw), 'header bytes:', size)
print('header:', raw[8:8 + size].decode().strip())
print('numbers:', raw[8 + size:].hex(' ', 4))
print('read back:', read_safetensors(path)['w'])
cd ~/xslm
mkdir -p lab
micro lab/safetensors.py
python3 -m lab.safetensorsOutput
file bytes: 128 header bytes: 104
header: {"__metadata__": {"format": "xslm"}, "w": {"dtype": "F32", "shape": [2, 2], "data_offsets": [0, 16]}}
numbers: cdcccc3d 0000003f 000000c0 00004040
read back: [[0.10000000149011612, 0.5], [-2.0, 3.0]]
model.safetensors is just a table of numbers. To use those numbers again, the person who downloads it needs three more things.
export.py gathers all of these files in the hub folder. The training data (data) and train.py are not uploaded. They are not needed to use the model, and the data needs its rights checked separately. Big models work the same way. A model repository on Hugging Face usually has config.json, tokenizer.json and *.safetensors together.
The model card is the README.md of the repository. It has a top part and a bottom part.
The body should cover at least five things: what it is, which files it has, how to run it, what data it was trained on, and what it cannot do (limitations). Being honest about the limitations matters. If you write that the answers are mostly random because it has 1,248 , nobody will use it the wrong way.
Anyone in the world can see and download a Public repository. Check three things before uploading.
You can upload as Private first to check, and switch to Public in the repository Settings once you are happy.
git was built to record the changed lines of text that has lines, such as source code. files like weights have no lines, so even a small change stores the whole file again. That is why Hugging Face rejects a push that puts binary files directly inside git, even small ones.
git lfs install is a one-time setting that connects LFS to git. After that you just git add, commit and push as usual. git lfs ls-files shows which files went to LFS.
Hugging Face does not accept your account password for git push. Instead, put a created under Settings → Access Tokens (a long string starting with hf_) where the password goes.
If you think a token has leaked, delete it right away and make a new one. A deleted token can do nothing.
A published model only means something if it runs on someone else's computer. The person downloading it does this.
With the same seed and settings, you get exactly the same answer as in this terminal. That is reproduction. Big models work the same way: the receiver reads the safetensors and runs it in their own (PyTorch, , and so on).
First sign up at huggingface.co and create a Public model repository named xslm-tiny at huggingface.co/new. Also create one token with Write permission under Settings → Access Tokens. Then type the below in order. Replace YOUR_ID with your Hugging Face username. When git push asks for Username, enter your username; when it asks for Password, paste the token instead of your password. The token is not shown on screen.
cd ~/xslm
micro export.py
sudo apk add git-lfs
git lfs install
git clone https://huggingface.co/YOUR_ID/xslm-tiny hub
python3 export.py
nano hub/README.md
cd hub
git config user.name "YOUR_ID"
git config user.email "you@example.com"
git config credential.helper cache
git add .
git commit -m "My first XSLM"
git pushwrite_safetensors writes the numbers one after another as 4-byte float32 values and puts a JSON header in front. read_safetensors does the reverse. export gathers the weights, settings, tokenizer and run code in the hub folder, then reads them back and shows the difference. --restore turns downloaded files back into checkpoints/model.json.
import argparse
import json
import shutil
import struct
from pathlib import Path
SOURCES = ['engine.py', 'model.py', 'tokenizer.py', 'generate.py',
'runner.py', 'xchat.py', 'export.py']
def write_safetensors(path, weights):
header, body = {'__metadata__': {'format': 'xslm'}}, b''
for name, matrix in weights.items():
rows, cols = len(matrix), len(matrix[0])
data = struct.pack('<%df' % (rows * cols),
*[v for row in matrix for v in row])
header[name] = {'dtype': 'F32', 'shape': [rows, cols],
'data_offsets': [len(body), len(body) + len(data)]}
body += data
head = json.dumps(header).encode('utf-8')
head += b' ' * (-len(head) % 8)
path.write_bytes(struct.pack('<Q', len(head)) + head + body)
def read_safetensors(path):
raw = path.read_bytes()
size = struct.unpack('<Q', raw[:8])[0]
header = json.loads(raw[8:8 + size])
body, weights = raw[8 + size:], {}
for name, info in header.items():
if name == '__metadata__':
continue
assert info['dtype'] == 'F32', 'only float32'
rows, cols = info['shape']
start, end = info['data_offsets']
flat = struct.unpack('<%df' % (rows * cols), body[start:end])
weights[name] = [list(flat[r * cols:(r + 1) * cols])
for r in range(rows)]
return weights
def export(folder):
saved = json.loads(Path('checkpoints/model.json').read_text())
folder.mkdir(exist_ok=True)
write_safetensors(folder / 'model.safetensors', saved['weights'])
config = {'model_type': 'xslm', 'vocab_size': len(saved['tokenizer']['vocab']),
'step': saved['step'], 'settings': saved['settings']}
(folder / 'config.json').write_text(json.dumps(config, indent=2))
(folder / 'tokenizer.json').write_text(
json.dumps(saved['tokenizer'], ensure_ascii=False), encoding='utf-8')
for name in SOURCES:
shutil.copy(name, folder / name)
loaded = read_safetensors(folder / 'model.safetensors')
gap = max(abs(a - b) for name in loaded
for row, old in zip(loaded[name], saved['weights'][name])
for a, b in zip(row, old))
count = sum(len(row) for matrix in loaded.values() for row in matrix)
print('parameters:', count, 'bytes:', (folder / 'model.safetensors').stat().st_size)
print('largest float32 gap:', '%.1e' % gap)
print('exported:', folder)
def restore(folder):
config = json.loads((folder / 'config.json').read_text())
state = {'settings': config['settings'],
'tokenizer': json.loads((folder / 'tokenizer.json').read_text(encoding='utf-8')),
'weights': read_safetensors(folder / 'model.safetensors')}
Path('checkpoints').mkdir(exist_ok=True)
Path('checkpoints/model.json').write_text(
json.dumps(state, ensure_ascii=False), encoding='utf-8')
print('restored: checkpoints/model.json')
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--to', default='hub')
parser.add_argument('--restore', metavar='FOLDER')
args = parser.parse_args()
if args.restore:
restore(Path(args.restore))
else:
export(Path(args.to))
This is the model card. The YAML between the --- lines at the top is information Hugging Face reads (license, language, tags, task type), and below it is the description people read. Replace YOUR_ID with your username, and if you changed the data, change the parameter count and the training data description to yours as well.
---
license: mit
language:
- en
tags:
- xslm
- tiny
- educational
pipeline_tag: text-generation
---
# XSLM tiny
A 1,248-parameter GPT that I typed from scratch and trained
inside my web browser, following https://sonub.com/ai/build
## Files
- model.safetensors: trained weights (float32)
- config.json: size and sampling settings
- tokenizer.json: character BPE vocabulary and merge rules
- engine.py, model.py, tokenizer.py, generate.py: the model code
- runner.py, xchat.py: HTTP Runner and terminal chat
- export.py: writes these files, and turns them back into a checkpoint
## How to run
Python 3 only. Nothing to install.
git clone https://huggingface.co/YOUR_ID/xslm-tiny
cd xslm-tiny
python3 export.py --restore .
python3 generate.py
## Training data
15 short English Q/A lines written for the course
(12 for training, 3 for validation).
## Limitations
For learning only. With 1,248 parameters and a few training
steps, the answers are mostly random. Do not use it for real work.
import argparse
import json
import shutil
import struct
from pathlib import Path
SOURCES = ['engine.py', 'model.py', 'tokenizer.py', 'generate.py',
'runner.py', 'xchat.py', 'export.py']struct is the that turns numbers into bytes and back, and shutil copies files. SOURCES is the list of source files needed to run the model. prepare.py and train.py, which are only for , are left out.
def write_safetensors(path, weights):
header, body = {'__metadata__': {'format': 'xslm'}}, b''
for name, matrix in weights.items():
rows, cols = len(matrix), len(matrix[0])
data = struct.pack('<%df' % (rows * cols),
*[v for row in matrix for v in row])
header[name] = {'dtype': 'F32', 'shape': [rows, cols],
'data_offsets': [len(body), len(body) + len(data)]}
body += data
head = json.dumps(header).encode('utf-8')
head += b' ' * (-len(head) % 8)
path.write_bytes(struct.pack('<Q', len(head)) + head + body)Each layer's is flattened into one row and turned into bytes with struct.pack. In the format <%df, < means little-endian, f is a 4-byte float32, and the number of values goes where %d is. The header records this layer's dtype and shape and its start and end position inside the body. Spaces are added so the JSON header length is a multiple of 8, and that length is written first as 8 bytes (<Q).
def read_safetensors(path):
raw = path.read_bytes()
size = struct.unpack('<Q', raw[:8])[0]
header = json.loads(raw[8:8 + size])
body, weights = raw[8 + size:], {}
for name, info in header.items():
if name == '__metadata__':
continue
assert info['dtype'] == 'F32', 'only float32'
rows, cols = info['shape']
start, end = info['data_offsets']
flat = struct.unpack('<%df' % (rows * cols), body[start:end])
weights[name] = [list(flat[r * cols:(r + 1) * cols])
for r in range(rows)]
return weightsThe reverse of writing. The first 8 bytes give the header length, and that many bytes are read as JSON. __metadata__ is not a layer, so it is skipped. data_offsets cuts out just that layer's bytes from the body, struct.unpack takes the numbers out, and they are split every cols values back into a matrix.
def export(folder):
saved = json.loads(Path('checkpoints/model.json').read_text())
folder.mkdir(exist_ok=True)
write_safetensors(folder / 'model.safetensors', saved['weights'])
config = {'model_type': 'xslm', 'vocab_size': len(saved['tokenizer']['vocab']),
'step': saved['step'], 'settings': saved['settings']}
(folder / 'config.json').write_text(json.dumps(config, indent=2))
(folder / 'tokenizer.json').write_text(
json.dumps(saved['tokenizer'], ensure_ascii=False), encoding='utf-8')
for name in SOURCES:
shutil.copy(name, folder / name)It reads checkpoints/model.json saved by training. The weights go to safetensors, the size, settings and training steps to config.json, and the vocabulary and merge rules to tokenizer.json. With ensure_ascii=False, characters such as Korean are stored as they are. Finally it copies SOURCES into hub.
loaded = read_safetensors(folder / 'model.safetensors')
gap = max(abs(a - b) for name in loaded
for row, old in zip(loaded[name], saved['weights'][name])
for a, b in zip(row, old))
count = sum(len(row) for matrix in loaded.values() for row in matrix)
print('parameters:', count, 'bytes:', (folder / 'model.safetensors').stat().st_size)
print('largest float32 gap:', '%.1e' % gap)
print('exported:', folder)It reads the file it just wrote with read_safetensors and finds the value that differs most from the original numbers. The difference comes from shrinking to float32, so around 1e-8 is normal. It also prints the parameter count and file size to show that hub was made correctly.
def restore(folder):
config = json.loads((folder / 'config.json').read_text())
state = {'settings': config['settings'],
'tokenizer': json.loads((folder / 'tokenizer.json').read_text(encoding='utf-8')),
'weights': read_safetensors(folder / 'model.safetensors')}
Path('checkpoints').mkdir(exist_ok=True)
Path('checkpoints/model.json').write_text(
json.dumps(state, ensure_ascii=False), encoding='utf-8')
print('restored: checkpoints/model.json')It gathers the settings from config.json, tokenizer.json and the weights in safetensors, and writes checkpoints/model.json in exactly the shape that load in generate.py reads. That is why generate.py, runner.py and xchat.py run the downloaded model without changing a single line.
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--to', default='hub')
parser.add_argument('--restore', metavar='FOLDER')
args = parser.parse_args()
if args.restore:
restore(Path(args.restore))
else:
export(Path(args.to))With no arguments it exports to hub, and --to gives another folder. Give --restore a folder and it rebuilds the checkpoint from the files there. Inside a downloaded repository, give a dot (the ), as in python3 export.py --restore .
Open huggingface.co/YOUR_ID/xslm-tiny. Is the model card on the first screen, and does model.safetensors in the Files tab have an LFS mark? Git clone it into another folder, run python3 export.py --restore . and python3 generate.py, and if you get the same answer as in this terminal, your model is published.
If git push is rejected because of binary files, you skipped git lfs install. Delete the folder with hub and start again from git lfs install. failed means you entered your password or a token with only Read permission. Refreshing the page removes hub/.git, so in that case too run rm -rf hub, then git clone and python3 export.py again. Once everything is uploaded, you may delete the token under Access Tokens.
Yes. A Private repository is visible only to you and the people you allow. Uploading works the same way; the only difference is that downloading also needs a token. When you have checked it, you can switch it to Public in the repository Settings.
To update it, train again, run python3 export.py, then git add, commit and push in hub. A new commit is added and older versions stay in the history. To delete it, press the Delete button at the bottom of the repository Settings. A deleted repository cannot be brought back.
Not as it is. reads only registered architectures such as and , and XSLM has a different layer layout and a character-based tokenizer. That is why the run code is uploaded too, and export.py --restore runs it with our own code. The safetensors file itself is a standard format, so the safetensors can open it and read it as arrays.