Blog
In total, I've written 11 articles on this site. Use the search below to filter by title, or pick a category.
Subscribe to Blog
No spam ever, only raw unfilered throughts and new blog post alerts.
AI Architecture
Take a night drive through Kimi K3 as an interactive 3D city: follow one token through Kimi Delta Attention, MLA, attention residuals and 896 experts.
Take the red pill and follow one token through DeepSeek V4-Pro in interactive 3D: compressed attention, hyper-connections, 384 experts and 61 layers.
Press start and watch a large language model finish a sentence in an interactive visualization: tokens, attention, the KV cache and mixture of experts.
How AI models think
––– views
How a machine that only predicts the next word ends up building world models, planning ahead, and doing things that look like thinking.
Exploring GPT-4: Part 1
––– views
Explain how you understand language and what are your core competencies as an AI model.