mii-llm, an open source AI lab, released Zagreus-0.4B-por, a compact bilingual Portuguese–English language model pretrained entirely from scratch.

The model has approximately 400 million parameters and is part of the Zagreus family, an ongoing experiment in building small open models focused on European languages.

Training was sponsored by Seeweb cloud provider and Regolo.ai, which provided the computing infrastructure used for the project.

The pretraining corpus was assembled from open datasets released through Hugging Face, including:

• FineWeb
• Portuguese FineWeb2
• Portuguese FinePDFs
• StarCoderData

Evaluation results

On the Portuguese versions of ARC, HellaSwag and MMLU, the final checkpoint achieved an average score of 0.3113.

Using Eduardo Garcia’s Portuguese evaluation harness, the 483k checkpoint scored:

• Zagreus-0.4B-por: 0.3230
• Qwen3-0.6B-Base: 0.2569

These results are encouraging for a model of this size, but benchmark performance is not the main reason for the release.

Zagreus-0.4B-por is an open base model intended to serve as a starting point for:

• instruction tuning
• domain-specific assistants
• educational tools
• research experiments
• lightweight edge and on-device applications

It is not presented as a finished assistant or production-ready product. It is an open foundation that others can inspect, fine-tune, modify and build upon.

We would be interested in feedback, independent evaluations and experiments from the Portuguese NLP and open-source communities.

Some useful links:

the recipe: https://github.com/mii-llm/zagreus-nesso-slm
the model: https://huggingface.co/mii-llm/zagreus-0.4B-por