Noema 1.5 2B · NoemaAI

Noema 1.5 2B is the second release in our 2B line. It starts from Noema 2B, which is post-trained from Qwen3.5-2B, and was trained to recover the broad knowledge Noema 2B gave up while keeping its strengths in mathematics, code and structured answers. The result follows instructions more precisely, holds constraints across turns better and reasons in fewer tokens.

No retrieval, external tools, inference-time answer repair or outside model calls were used to produce the reported results; the improvements live in the weights.

Results against stock Qwen3.5-2B

We froze the release candidate and ran it once against stock Qwen3.5-2B, with identical prompts, generation settings and graders. Point estimates are reported even where they were not statistically resolved, and losses are included.

Final evaluation, BF16 inference. Percentages unless noted. IFEval and the all-three-turn Multi-IF gains are statistically significant (p = 0.000475 and p = 4.44e−8). HumanEval+ and thinking accuracy are favorable but not statistically resolved. MMLU-Pro was the preregistered knowledge-retention endpoint and passed its non-inferiority gate.
Benchmark	Noema 1.5 2B	Qwen3.5-2B	Difference
IFEval, prompt strict (541 prompts)	72.09	65.06	+7.02
Multi-IF, all three turns (4,501 conversations)	17.88	14.77	+3.11
Multi-IF, mean per turn	35.84	33.49	+2.36
HumanEval+, pass@1 (164 problems)	53.66	50.61	+3.05
Thinking accuracy (300 prompts)	68.67	67.33	+1.33
Thinking tokens, mean per completion	6,895	7,969	−13.5%
MMLU-Pro (11,332 questions)	53.17	53.88	−0.71
GPQA Diamond (198 questions)	40.40	45.45	−5.05
IFBench, prompt loose (300 prompts)	27.33	31.00	−3.67

The clearest gains are in following instructions. Strict prompt-level IFEval rises by 7 points, and the share of three-turn conversations that satisfy every turn rises from 14.8% to 17.9%. Reasoning traces are 13.5% shorter, with a favorable accuracy point estimate.

Progress from Noema 2B

During development, paired gates against the exact Noema 2B starting checkpoint guided model selection. MMLU-Pro on a matched 700-question set rose from 50.14 to 53.00, HumanEval+ from 50.00 to 53.66, strict IFEval from 70.24 to 73.01, and a verified mathematics set from 78.00 to 92.00. Because these gates informed selection, they are not part of the final evaluation above.

Where it falls short

GPQA Diamond and IFBench finished below stock Qwen3.5-2B in the final point estimates.
HumanEval+ and thinking accuracy were favorable but not statistically resolved.
Evaluation was primarily English and text only; context was independently validated to 24,576 tokens.

Specifications

About 2B parameters, starting from Noema 2B (foundation: Qwen3.5-2B).
Hybrid architecture: Gated DeltaNet and attention, 24 layers, hidden size 2,048.
262K-token native context, independently evaluated to 24,576 tokens.
Non-thinking and thinking modes. Primary language: English.
Transformers, GGUF (BF16 to Q4_K_M) and MLX (BF16, 8-, 6- and 4-bit) builds.

How to run it

Download Noema 1.5 2B into the Noema model library and choose a device-aware runtime preset, or run the GGUF build with llama.cpp, as its model card shows:

llama.cpp

llama-cli \
  -hf NoemaAI-labs/Noema-1.5-2B-GGUF:Q4_K_M \
  -cnv \
  -rea off \
  --temp 0

Availability

Noema 1.5 2B is available on Hugging Face. It is an open-weight release, not an open-source release: the weights are publicly downloadable, but no open-source license is granted. Full results are on the model page.