CHAPTER II. Toktumi 2.0. Second Training Run.

In this part, I describe the second stage of the Toktumi project, which is educational, research-oriented, and non-commercial. I introduce improvements based on the experience and lessons learned from training Toktumi 1.0.

As a reminder: In the previous stage, I built and trained a Polish-language GPT-2 Medium model from scratch, using 3 115 002 text samples that I had collected and cleaned myself. I prepared my own tokenizer, carried out the training, and tested the quality of the model's responses. As a result, Toktumi 1.0 learned to generate grammatically correct text, but it often struggled to recognize instructions and provide correct answers.

For more details, see Part 1.

1. Why Train the Model from Scratch Again?

I decided to train a new model from scratch. The goal was to implement two key recommendations:

1. Remove internet comments from the training dataset, which I expected would eliminate chaotic and low-quality content.
2. Add Question–Answer (QA) and Instruction–Answer (IA) datasets to help the model learn how to interact with users and generate coherent responses to specific prompts and instructions.

Fine-Tuning or Training from Scratch Again?

Instead of fine-tuning (continuing the training of an already trained model), I decided to train a new model entirely from scratch. I was concerned that fine-tuning would not eliminate the problems caused by the original training dataset. Training from scratch makes it possible to build a model using only a cleaned and properly prepared text corpus, providing greater control over the quality of the result.

Since I had already completed the entire model-training process once, the second iteration was simpler and less time-consuming. Many steps could be repeated with only minor modifications, and changes were needed only where necessary. As before, the biggest challenge was preparing new texts for training the model, this time in the form of question–answer and instruction–answer pairs.

2. Planning the Next Stages

2a. Collect new data - prepare Question–Answer (QA) and Instruction–Answer (IA) datasets.

2b. Prepare the complete training dataset.

2c. Train from scratch - carry out a full training run using the GPT-2 Medium architecture and the improved text dataset.

2d. Evaluate model quality - stage 1 - measure checkpoint perplexity.

2e. Evaluate model quality - stage 2 - prompting and human evaluation.

2f. Finish training at the selected checkpoint.

2a. Data Collection. Question–Answer (QA) and Instruction–Answer (IA) Datasets

I started this stage in early 2025. At the time, I could not find extensive instruction datasets in Polish online, although such datasets existed in other languages, mainly English. Examples included databricks-dolly-15k (15,000 examples, English), HuggingFaceH4 helpful-instructions (150,000 examples, English), OIG: Open Instruction Generalist (44 million examples, English), and Aya Dataset (513 million examples in over 100 languages, but relatively few in Polish).

In this situation, an attractive option was to generate such data myself, using existing LLMs. This approach is often referred to as generating synthetic data. More on this topic: [Article].

I decided to carry out two experiments:

• Generate data locally on my home PC using Mistral-Nemo-12B-Instruct-2407, downloaded from Hugging Face, with prompts managed through Python.
• Generate data using the Gemini API (Google's model). This allowed me not only to collect data, but also to practice integrating Python with an external API and compare the quality of the results. I used gemini-2.5-flash.

Both approaches had their advantages and limitations. Running Mistral locally gave me full control over the process and made me independent of external services. The downside was the hardware limitations of my GeForce RTX 3060, which meant long generation times and significant energy consumption - greater than during the model training itself.

Gemini, on the other hand, runs in the cloud. It is faster, easier to use, and produces more polished responses in Polish. However, this approach also has its limitations and API-imposed quotas, which made large-scale text generation difficult. Paid access was an alternative, but at this scale the costs would have been too high for the project's limited budget.

INSTRUCTION DATASETS GENERATED LOCALLY (MISTRAL NEMO 12B)

To generate Question–Answer (QA) and Instruction–Answer (IA) datasets, I used Mistral-Nemo-12B-Instruct-2407, running locally on my home PC.

How the code works: My working directory contained text files (.txt) with articles on various topics, previously collected from fakt.pl and polsatnews.pl. I passed each text to the model together with a detailed prompt specifying the required output format (questions and instructions with their answers). This was also an interesting exercise in prompt engineering. With Mistral 12B, refining the prompt was crucial - only extensive testing produced satisfactory results.

Along with the prompt, the model receives the following instructions:

• generate up to five questions with different structures and lengths, together with their answers,
• limit closed-ended "yes/no" questions (the model tended to overuse this format),
• generate different types of instructions related to the article,
• use only correct, fluent Polish,
• avoid additional comments and formatting marks (such as asterisks or hyphens), although Mistral still inserted them occasionally.

This process made it possible to automatically generate ready-to-use sets of text pairs. Below is a section of the code used to process a sample article.

Example: The original text being converted into a set of questions and instructions.

The code used to generate these datasets reads an article, passes it to the model along with the prepared prompt, and then generates responses. Due to hardware limitations, I also used 4-bit quantization. More on quantization.

import random
import torch
from time import time
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

# Model i plik źródłowy
model_id = "Mistral-Nemo-12B-Instruct-2407"
input_file = "sample.txt"

# Kwantyzacja 4-bitowa
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

# Wczytanie modelu i tokenizera
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    quantization_config=quant_config
).eval()

# Wczytanie artykułu
with open(input_file, "r", encoding="utf-8") as f:
    text = f.read().strip()

# Losowanie czasowników do przykładów instrukcji
verbs = [
    "Opisz", "Streść", "Wyjaśnij", "Wymień", "Znajdź",
    "Podaj", "Napisz", "Wylicz", "Przeanalizuj",
    "Określ", "Uzasadnij", "Przedstaw", "Oceń",
    "Scharakteryzuj", "Wskaż", "Zdefiniuj", "Porównaj",
    "Zaproponuj", "Uporządkuj", "Powiedz", "Opowiedz"
]

random.shuffle(verbs)
examples = ", ".join(verbs[:7])

# Prompt — pełna treść przedstawiona poniżej
prompt_template = """treść promptu"""
prompt = prompt_template.format(examples=examples, text=text)

# Przygotowanie wejścia dla modelu
messages = [
    {"role": "system", "content": ""},
    {"role": "user", "content": prompt.strip()}
]

chat_input = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Generowanie pytań, instrukcji i odpowiedzi
start_time = time()

with torch.inference_mode():
    outputs = model.generate(
        chat_input,
        max_new_tokens=1024,
        do_sample=True,
        temperature=0.4,
        top_p=0.7
    )

# Dekodowanie samej wygenerowanej odpowiedzi
generated_ids = outputs[0][chat_input.shape[-1]:]
result = tokenizer.decode(generated_ids, skip_special_tokens=True).strip()

print(result)
print(f"Czas generowania: {time() - start_time:.2f} s")

The key element is the prompt itself (written in Polish, which Mistral can handle):

Zadaj maksymalnie 5 pytań, na które odpowiedzi znajdują się w treści. Pytania muszą być różnorodne pod względem struktury i długości. Nie wolno zadawać więcej niż jednego pytania, które zaczyna się od „Czy”. Nie zadawaj więcej niż jednego pytania, którego odpowiedź mogłaby być „tak” lub „nie”. Zabronione jest powtarzanie pytań. Pod pytaniem zawsze napisz odpowiedź. Odpowiedź ma być tak szczegółowa, jak to potrzebne.

Następnie stwórz kilka przykładowych tekstów instrukcji w kontekście treści tego tekstu (format imperativus, np. {examples}). To tylko przykłady, wybierz takie, które najbardziej będą pasować do tekstu. Staraj się, aby teksty instrukcji były różnorodne i różniły się między sobą. Zabronione jest powtarzanie instrukcji. Pod instrukcją zawsze napisz odpowiedź. Odpowiedzi powinny być różnorodne.

Uwagi do całego tekstu, który stworzysz: Pisz tylko i wyłącznie w języku polskim. Pisz płynną, poprawną polszczyzną. Nie oznaczaj tekstu dodatkowymi komentarzami od siebie. Nigdy nie stosuj gwiazdek '*', '**' ani myślników '-' w celu wyróżnienia tekstu. Chcę mieć czysty tekst gotowy do użycia: pytanie-odpowiedź, instrukcja-odpowiedź. Nie dodawaj nic więcej. Zapisz wszystko jako ciąg pytań i odpowiedzi oddzielonych pustymi liniami.

Tekst: {text}

English translation of the prompt:

Ask a maximum of 5 questions whose answers can be found in the text. The questions must vary in structure and length. Do not ask more than one question beginning with “Czy”. Do not ask more than one question that could be answered with “yes” or “no”. Repeating questions is prohibited. Always write the answer below each question. The answer should be as detailed as necessary.

Then create several examples of instructions related to the text (imperative form, e.g. {examples}). These are only examples; choose those that best fit the text. Try to make the instructions varied and different from one another. Repeating instructions is prohibited. Always write an answer below each instruction. The answers should be varied.

Notes for the entire text you produce: Write only in Polish. Use fluent, correct Polish. Do not add any comments or labels of your own. Never use asterisks '*', '**' or hyphens '-' to emphasize text. I want clean text ready to use: question–answer, instruction–answer. Do not add anything else. Write everything as a sequence of questions and answers separated by blank lines.

Text: {text}

The prompt had to include these details because, as the tests showed, the model tended to take the path of least resistance and generate repetitive structures, often beginning with “Czy”. That is why I limited the number of such question forms in the prompt.

In addition, when generating instructions, I used a simple trick involving the {examples} variable. The model tended to produce monotonous, repetitive instructions. I selected 21 verbs that worked best for this task. For each article, I then randomly selected 7 of them and supplied them to the model as examples in the prompt. Tests showed that this was the most effective way to increase the variety of generated instructions, both with the locally run Mistral model and with the Gemini API.

The 21 Polish verbs (translated into English) were: Describe, Summarize, Explain, List, Find, Provide, Write, Enumerate, Analyze, Specify, Justify, Present, Evaluate, Characterize, Identify, Define, Compare, Suggest, Organize, Say, Tell me about.

MISTRAL – RESULTS:

The code successfully converted the source text into a set of questions and instructions with corresponding answers. However, generation was relatively time-consuming — on my graphics card, it took an average of about 1 minute per article.

With 4-bit quantization, the generated texts occasionally contained minor linguistic errors, such as “Kto był Adam West?” instead of “Kim był Adam West?”. (Both translate as “Who was Adam West?” in English, but the first uses the wrong grammatical form in Polish.) Such constructions occurred relatively often with the local Mistral model, affecting the quality of the training dataset. Despite this, the method served its purpose well.

The Mistral test was a valuable experiment, demonstrating both the potential and the limitations of working with large locally run models.

INSTRUCTION DATASETS GENERATED ONLINE USING THE GEMINI API (GOOGLE)

For the second experiment, I used Gemini 2.5 Flash, a model developed by Google that was popular at the time (2025) and accessible through an API. This approach allows the model to be integrated with Python and text generation to be automated, including batch processing of many files.

As with Mistral, I converted source articles into sets of questions and instructions with answers. The difference was that all the computational work was handled by the service provider.

Gemini 2.5 Flash proved significantly more effective than the locally run Mistral 12B. It generated texts in more fluent Polish, interpreted instructions better, and followed the required response format more consistently.

GEMINI – RESULTS:

Gemini 2.5 Flash generated content more than twice as fast (around 30 seconds per article) and at a higher quality. It also handled more complex instructions more effectively, such as inserting delimiters.

Using an API, however, has its limitations, especially restrictions on the number of requests and generated tokens. When processing large numbers of articles, this was a significant obstacle that did not arise with the locally run model.

Ultimately, I generated most of the texts locally using the Mistral model.

Example: An excerpt from the training dataset. Cleaned question–answer (QA) and instruction–answer (IA) pairs, ready for training.

After more than 40 days of generating data in both environments (local and cloud-based), I managed to collect 616 279 text pairs in the form of Question–Answer (340 447) and Instruction–Answer (275 832) records.

Generating synthetic data at scale is costly and time-consuming — it requires substantial computing power, whether using a local GPU or an API service.

Web scraping is much faster and cheaper, but it does not always provide texts of the required quality and structure.

Another option would have been automatically translating existing instruction datasets from English. I decided against this approach, however, because I considered the results of my experiments so far sufficient.

2b. Preparing the New Training Dataset

For the new training dataset, I reused texts previously collected from the following sources: Wikipedia, Pudelek, Polsat News, books, and Fakt. However, I excluded Wykop and jokes, which, as earlier tests had shown, negatively affected the quality of the model's responses. The model would often respond in the style of internet comments or produce answers that had little connection to reality.

In addition, I used web scraping to collect 67 846 new articles from Fakt, published up to August 12, 2025 (the cutoff date for this dataset). The purpose was to update the model's knowledge of current events, such as Donald Trump taking office as President of the United States and Karol Nawrocki becoming President of Poland. The tests showed how important it is to keep training data up to date. This is one of the challenges facing language model developers. An LLM's knowledge needs to keep pace with a constantly changing world; otherwise, its answers about current events quickly become outdated.

The most important change, however, was adding 616 279 synthetically generated question–answer and instruction–answer pairs to the training dataset, prepared during the earlier experiments with Mistral and Gemini.

The result was a hybrid training dataset combining original source texts with synthetically generated question–answer and instruction–answer pairs. This allowed the model to learn from both natural texts and examples of direct interaction with a user.

Sources and text datasets used to train Toktumi 2.0:

Despite adding a large instruction dataset, removing the Wykop data (previously the largest category of texts) meant that the training dataset shrank by approximately 25% in terms of the number of text samples and by approximately 40% in terms of characters and tokens.

As it later turned out, this step also contributed to improving the quality of Toktumi 2.0.

Comparison of training dataset sizes: Toktumi 1.0 | Toktumi 2.0.

2c. Training the Model

Next, I followed the same steps as when training Toktumi 1.0:

• I reduced the character set to 256 characters (by converting UTF-8 → ANSI → UTF-8).
• I separated text samples using the delimiter <|endoftext|>.
• I set aside a validation file from the training data (a text file of approximately 10 MB).
• I prepared a new tokenizer based on the new training dataset (vocab_size = 50 000). It is important to understand the consequences of this step: each tokenizer has its own vocabulary and token IDs, so the new tokenizer was not compatible with the Toktumi 1.0 tokenizer. This was a valuable lesson — from that point onward, I kept the Toktumi 2.0 tokenizer for all subsequent stages of model development.
• I created an empty GPT-2 Medium model, using the same method as before.
• I started training, this time setting the number of epochs to 8 (bearing in mind that Toktumi 1.0 training had ended at around 6.44 epochs).

Training Toktumi 2.0 on a smaller dataset and with fewer planned epochs meant that the entire process involved 829 048 steps, and completing 8 epochs took approximately 11.5 days (excluding technical interruptions for restarting the computer and resuming training from checkpoints) — compared with approximately 15 days (6.44 epochs) for Toktumi 1.0.

2d. Model Evaluation. Stage 1: Perplexity.

As in the first stage of the project, I evaluated the model in two stages. First, I measured perplexity for successive checkpoints using the evaluation code I had prepared earlier. The charts below show the results for both models.

The learning curves of the two models differ somewhat. For Toktumi 1.0, perplexity decreased gradually, reaching approximately 26. In contrast, Toktumi 2.0 showed a sharper decline and quickly stabilized at around 19. This pattern suggests that Toktumi 2.0 achieved a better fit to its data and produced more consistent predictions.

However, it is important to remember that the two models were evaluated on different validation datasets, so the comparison should be treated as approximate.

Comparison of perplexity in Toktumi 1.0 and Toktumi 2.0:

2e. Model Evaluation. Stage 2: Prompting.

The second stage of evaluation involved testing Toktumi 2.0 on a set of 150 prompts, the same ones I had previously used to evaluate Toktumi 1.0 (see Part 1).

I automated the testing process with Python code that ran the prompts on successive checkpoints and saved the responses. I then evaluated the results manually.

Because of the large number of responses, I also used AI to help analyze the results. The assessments were not always clear-cut, but after several rounds of comparison, I was able to identify the best checkpoint.

As before, I used greedy decoding (top_k = 1), which ensured that the model always generated the same, most probable response. This made it possible to keep the testing conditions consistent.

The full list of 150 prompts and model responses -> VIEW.

2f. Ending the Training Run

Ultimately, I selected checkpoint-787500, saved at around 7.60 epochs, as the best checkpoint.

I used it to prepare the final Toktumi 2.0 model, together with its tokenizer.

3. Final Model Evaluation

In the final step, I carried out a final evaluation and comparison of Toktumi 1.0 and Toktumi 2.0. To ensure a consistent comparison, I developed and applied a single evaluation methodology. I classified each response in two steps:

STEP 1 — Instruction Following (IF)

Does the model correctly recognize the structure of the prompt (Instruction Following, IF)?

For example, when given a question, does it attempt to answer it as a question? When given an instruction, does it respond appropriately to the instruction? And when given an incomplete sentence, does it complete that sentence?

Toktumi 1.0 example:

Prompt 05: Wymień trzy warzywa. [EN: Name three vegetables.]

Answer: Wymień trzy warzywa. Te z nich są najbardziej wymagające. [EN: Name three vegetables. These are the most demanding ones.]

In this example, Toktumi 1.0 failed to recognize the structure of the instruction. Instead of following it and naming three vegetables, it continued the text as though extending the prompt. I classified such responses as NO.

Toktumi 2.0 example:

Prompt 05: Wymień trzy warzywa. [EN: Name three vegetables.]

Answer: - Ziemniaki, pomidory i ogórki. (np. ziemniaki, papryka, cebula). [EN: - Potatoes, tomatoes, and cucumbers. (e.g., potatoes, bell peppers, onions).]

In this example, Toktumi 2.0 correctly recognized the instruction and named three vegetables. I labeled such responses YES.

STEP 2 — Factuality (FCT)

In the next step, I checked whether the answer was correct, factually accurate, and made sense (Factuality, FCT).

I classified responses containing hallucinations, false information, or internal contradictions negatively, marking them as HAL.

I applied lenient evaluation criteria — partially correct responses could also receive a YES rating if the key part of the answer was correct.

Toktumi 1.0 example:

Prompt 13: Mężczyzna to osoba płci [EN: A man is a person of the ... sex.]

Answer: Mężczyzna to osoba płci żeńskiej. [EN: A man is a person of the female sex.]

In this example, Toktumi 1.0 recognized the language structure and completed the sentence / gave an answer (STEP 1 → YES). However, the answer was factually incorrect. I labeled such responses HAL. For simplicity, I include them in the same category as hallucinations.

Toktumi 2.0 example:

Prompt 13: Mężczyzna to osoba płci [EN: A man is a person of the ... sex.]

Answer: Mężczyzna to osoba płci męskiej, która nie jest związana z żadną partią polityczną. [EN: A man is a person of the male sex who is not affiliated with any political party.]

In this example, Toktumi 2.0 correctly recognized the language structure and completed the sentence / gave an answer (STEP 1 → YES). In addition, the answer was consistent with real-world knowledge in the part about sex, while I treated the extra statement about having no political affiliation as incidental and not significantly affecting the correctness rating. I took a lenient approach, focusing on the most important parts of the response rather than making the criteria overly strict. I also classified such partially correct responses as YES.

Toktumi 1.0 example:

Prompt 62: Czekolada jest [EN: Chocolate is]

Answer: Czekolada jest zdrowa, ale niezdrowa. [EN: Chocolate is healthy, but unhealthy.]

In this case, Toktumi 1.0 gave an internally contradictory answer — it contained two mutually exclusive statements. I treated this type of response as an example of hallucination and also labeled it HAL.

This is how I arrived at a comparison of the response quality of both models.

Toktumi 1.0 Results

Toktumi 2.0 Results

FINDINGS

• Toktumi 2.0 produces higher-quality responses than version 1.0.

Conclusion: Training a new model from scratch was the right decision, and the improvements I introduced led to a clear increase in response quality.

• The greatest improvement was in instruction following (IF). The number of cases in which the model correctly recognized the instruction rose from 118 to 144. At the same time, the number of correct answers increased from 85 to 102, an improvement of approximately 20%.

Conclusion: Training Toktumi 2.0 on Question–Answer (QA) and Instruction–Answer (IA) datasets was effective and improved its ability to recognize and follow instructions.

• Toktumi 2.0 greatly reduced the problem of responses resembling internet comments — the model was less likely to imitate chaotic, judgmental user language and much more likely to act like a friendly assistant.

Conclusion: Removing Wykop comments from the training dataset was the right decision and improved the quality of generated responses. This shows that the quality of training data can matter more than its quantity.

ADDITIONAL OBSERVATIONS

Toktumi 2.0 handles instruction-style prompts significantly better than version 1.0.

[Toktumi 1.0]

Prompt 08: Podaj przykład emocji. [EN: Give an example of an emotion.]

Answer: Komentarz: Nie ma to jak wmawiać ludziom, że są lepsi od innych. [EN: Comment: Nothing beats convincing people that they are better than others.]

[Toktumi 2.0]

Prompt 08: Podaj przykład emocji. [EN: Give an example of an emotion.]

Answer: Przykładem emocji może być radość, radość lub smutek, gdy ktoś jest szczęśliwy i cieszy się z tego, co ma. [EN: An example of an emotion could be joy, joy or sadness, when someone is happy and enjoys what they have.]

Of the 14 instructions in the test set (such as "give..." or "name..."), Toktumi 1.0 responded correctly (IF=YES) in only 1 out of 14 cases, while Toktumi 2.0 did so in 13 out of 14.

In other words: in terms of instruction following, Toktumi 2.0 shows enormous progress in recognizing and carrying out instructions, compared with Toktumi 1.0's almost complete inability to do so.

Both models still struggled with factual knowledge, even when asked very simple questions.

The following pencil example illustrates this well.

[Toktumi 2.0]

Prompt 91: Do czego służy ołówek? [EN: What is a pencil used for?]

Answer: Ołówek to jeden z najpopularniejszych przedmiotów codziennego użytku. Jest niezbędny do prawidłowego funkcjonowania organizmu, a także do pracy fizycznej i psychicznej. Jego używanie jest bardzo proste, jednak nie każdy wie, jak się do tego zabrać. Jak się go pozbyć? [EN: A pencil is one of the most common everyday objects. It is essential for the proper functioning of the body, as well as for physical and mental work. Using it is very simple, but not everyone knows how to go about it. How do you get rid of it?]

Interestingly, the Wikipedia definition of the word "pencil" was present in the training datasets of both models. Even so, the model was unable to answer such a simple question correctly.

Experiment: After increasing the top_k parameter from 1 to 2 and introducing an element of randomness, the model generated an answer much closer to the expected definition.

Prompt: Do czego służy ołówek? [EN: What is a pencil used for?]

Answer: Ołówki to bardzo praktyczne i uniwersalne narzędzia do pisania. Są one nie tylko ozdobą, ale także ułatwiają pisanie tekstów oraz ich czytanie w różnych miejscach na świecie - od Londynu po Los Angeles czy Chicago. [EN: Pencils are very practical and versatile writing tools. They are not only decorative objects, but also make it easier to write and read texts in various places around the world - from London to Los Angeles or Chicago.]

Conclusion: The mere presence of information in the training dataset does not guarantee that the model will select it as the most obvious answer to a prompt (in greedy decoding mode). The experiment also showed how strongly generation hyperparameters and prompting techniques can influence the responses produced.

SUMMARY

In this part, I described the second stage of Toktumi's development (Toktumi 2.0), in which I applied the experience and lessons learned from the first training run (Toktumi 1.0). The most important change was rebuilding the training dataset — removing problematic internet comments and introducing 616 279 synthetically generated question–answer and instruction–answer pairs, prepared using the Mistral and Gemini models.

The result was a hybrid training dataset containing 2 336 162 text samples, combining natural source texts with examples of interaction with a user. I used it to create a new tokenizer and train a new GPT-2 Medium model from scratch, on my home graphics card.

The results confirmed that the changes were moving in the right direction — in the 150-prompt test, the number of correct answers rose from 85 to 102, and the number of correctly recognized instructions increased from 118 to 144.

The experiment demonstrated the importance of the quality and structure of training data, rather than just its quantity. Despite having a smaller text dataset, Toktumi 2.0 handled instructions much better and was less likely to imitate the chaotic style of internet comments.

The biggest unresolved problem was factual knowledge. The model still frequently hallucinated and struggled to give correct answers even to simple questions. I therefore decided it was worth taking the next step — rather than training another model from scratch, continuing to train Toktumi 2.0 on a new, synthetically prepared dataset of short questions, instructions, and dialogues.

You can read more about the model's development in Part 3.

Toktumi 2.0 on Hugging Face Spaces

You can try Toktumi 2.0 on Hugging Face Spaces:

https://huggingface.co/spaces/miczac/toktumi-2.0-polish-gpt2

NEXT PART