VIDRAFT Open-Sources Aether-7B-5Attn: A 7B Foundation Model with Training Code and Data
TL;DR: VIDRAFT, a Korean Pre-AGI AI startup, has publicly released Aether-7B-5Attn, a 7-billion-parameter foundation model alongside its training code and training data. The "5Attn" designation signals a distinct architectural choice in how attention is structured within the model. Developers gain full transparency into the training pipeline — not just the weights — making this a meaningful open-science release for the ML engineering community.
What it is
Aether-7B-5Attn is VIDRAFT's first publicly released foundation model, a large language model in the 7B parameter class. What distinguishes this release from a typical weights-only drop is the scope of what's being shared:
- Model weights — the trained Aether-7B-5Attn checkpoint
- Training code — the code used to train the model, enabling reproducibility and fine-tuning research
- Training data — the dataset(s) used during training, giving researchers insight into the data composition behind the model's behavior
The model name itself is informative: 7B refers to the approximate parameter count, and 5Attn indicates a specific attention-related architectural configuration (five attention heads or attention layers of a particular design — the exact semantics are defined in the released code and model card, so engineers should consult those directly rather than assume).
This is a foundation model release, meaning it is a base model intended as a starting point for downstream fine-tuning, instruction tuning, or further research — not a chat-tuned or RLHF-aligned assistant out of the box.
How it works
At a conceptual level, Aether-7B-5Attn follows the transformer-based autoregressive language modeling paradigm common to modern large language models. The architectural highlight is the 5Attn component, which represents VIDRAFT's design choice around attention mechanisms within the model.
Attention mechanisms are the core of how transformer models relate tokens to one another across a sequence. Variations in how many attention heads, attention layers, or attention grouping strategies (such as grouped-query attention or multi-query attention) are used can significantly affect:
- Inference throughput — fewer or more efficient attention heads can reduce KV cache memory and speed up decoding
- Training stability — attention architecture choices interact with depth, width, and learning dynamics
- Quality-efficiency tradeoffs — different attention configurations hit different points on the parameter-efficiency curve
Because VIDRAFT is releasing the training code alongside the weights, engineers can inspect exactly what 5Attn means in implementation — the kind of ground-truth transparency that is rarely available in commercial model releases.
The release of training data further allows the community to audit data composition, identify domain coverage, and understand what the model has and hasn't been exposed to — critical context for anyone planning to adapt the model for a specialized domain.
Benchmarks & results
The source article does not provide specific public benchmark numbers (e.g., scores on MMLU, HellaSwag, ARC, or HumanEval) for Aether-7B-5Attn. Qualitatively, the release is framed as a foundation model contribution from VIDRAFT, with the emphasis placed on openness and reproducibility rather than headline benchmark competition.
Engineers evaluating the model should:
- Refer to the official model card on Hugging Face (once published) for any evaluation results VIDRAFT has reported
- Run standard open LLM evaluation harnesses (e.g.,
lm-evaluation-harnessby EleutherAI) against their own task distributions - Treat the published training data as a signal for expected domain strengths and gaps
No benchmark numbers are reproduced here because none were reported in the source — inventing figures would be misleading.
How to try it
The source article confirms a public release of the model, training code, and data. VIDRAFT's standard public channels include Hugging Face and GitHub. Based on the public nature of this release:
- Check VIDRAFT's Hugging Face organization for the Aether-7B-5Attn model card, weights, and dataset repository
- Check VIDRAFT's GitHub for the training code repository
Since specific repository URLs, Hugging Face model IDs, or API endpoints were not confirmed in the source article, no commands are included here to avoid pointing developers to incorrect paths. Visit VIDRAFT's official channels directly to get the canonical links.
Once you have the correct Hugging Face model ID, a typical download would look like:
# Replace <org>/<model-id> with the actual published model identifier from VIDRAFT's HF page
huggingface-cli download <org>/<model-id>
FAQ
Q: What does "5Attn" actually mean in the model name — is it five attention heads, five attention layers, or something else?
A: The source article does not define the exact semantics of "5Attn" beyond it being an architectural designation. The released training code is the authoritative reference — inspect the model configuration file in the repository for the precise definition.
Q: Is this a base model or an instruction-tuned/chat model?
A: Aether-7B-5Attn is described as a foundation model, which in standard ML terminology means a base pretrained model. It is not reported to be instruction-tuned or RLHF-aligned. Developers who need a conversational or instruction-following variant would need to fine-tune it themselves.
Q: Can I use the training data for my own pretraining runs?
A: VIDRAFT is releasing the training data as part of this open release, but the specific license governing the data and weights is not detailed in the source article. Always check the license file in the repository before using released artifacts in a commercial or redistributed product.
Originally reported by 조선비즈 (2026-07-20) — source article.
