How to Build a Large Language Model from Scratch? Toktumi 1.0 – Part 1/3
Toktumi 1.0 – a Polish-language LLM based on GPT-2. From web scraping to a working model.
Series
How to Build a Large Language Model from Scratch? Toktumi (GPT-2)
A three-part project documenting the development of Toktumi, a Polish language model based on GPT-2. From data collection and training from scratch to dataset improvements, fine-tuning, and model evaluation.
Posts in this series
How to Build a Large Language Model from Scratch? Toktumi 2.0 – Part 2/3
Toktumi 2.0 – a Polish-language LLM based on GPT-2. How to learn from the first training and do it better.
How to Build a Large Language Model from Scratch? Toktumi 3.0 – Part 3 (The Final Part)
Toktumi 3.0 – a Polish-language LLM based on GPT-2. Fine-tuning, 100% synthetic training data. Conclusion: LESS CAN BE MORE.