Noema on iPhone answering a question from an on-device dataset
NewReusing Model Probabilities for New Queries

Small models,private AI.

NoemaAI researches small language models, releases them as open weights, and builds Noema, a private AI app that runs them on iPhone, iPad, Mac and Apple Vision Pro.

  • Models 3 open-weight releases
  • Research 1 preprint
  • Platforms iPhone · iPad · Mac · Vision Pro

Models

Open-weight models, measured against their Qwen base.

Skip to research

  1. Size
    4B
    Built on
    Qwen3.5-4B
    License
    Open weights
    Hugging Face (Noema 1.5 4B Deep weights, opens in a new tab)

    Thinks longer on hard problems. The more accurate of the two 4B styles.

    Selected results against Qwen3.5-4B. Paired against stock Qwen3.5-4B on the same prompts and settings. Under Qwen's preset it trails Qwen3.5-4B on AIME and HMMT. Loop rates come from a stress test of hard prompts, not everyday use.

    Full results
    • GPQA Diamond

      +15.6 pts

      Graduate-level science · Qwen's preset, 32,768 tokens, where a third of Qwen3.5-4B's answers are unfinished

      Noema 1.5 4B Deep
      74.7%
      Qwen3.5-4B
      59.1%
    • Tokens per answer

      ~40% fewer

      Mean over AIME 2026, HMMT Feb 2026, LiveCodeBench v6 and GPQA Diamond · Qwen's preset · same mean accuracy (67.2% vs 66.4%) · lower is better

      Noema 1.5 4B Deep
      22.1K
      Qwen3.5-4B
      37.4K
    • Repetition loops

      −51.1 pts

      1,000 hard prompts · greedy, 32,768 tokens · lower is better

      Noema 1.5 4B Deep
      13.9%
      Qwen3.5-4B
      65.0%
  2. Size
    4B
    Built on
    Qwen3.5-4B
    License
    Open weights
    Hugging Face (Noema 1.5 4B Fast weights, opens in a new tab)

    Answers in a fraction of the tokens and almost never gets stuck. Suits phones.

    Selected results against Qwen3.5-4B. Paired against stock Qwen3.5-4B on the same prompts and settings. About 16 points behind Qwen3.5-4B on AIME and HMMT under Qwen's preset. Loop rates come from a stress test of hard prompts, not everyday use.

    Full results
    • Repetition loops

      −63.9 pts

      1,000 hard prompts · greedy, 32,768 tokens · lower is better

      Noema 1.5 4B Fast
      1.1%
      Qwen3.5-4B
      65.0%
    • Repetition loops, 4-bit

      −46.2 pts

      Same prompts · Q4_K_M, llama.cpp default sampling, 32,768 tokens · lower is better

      Noema 1.5 4B Fast
      0.6%
      Qwen3.5-4B
      46.8%
    • GPQA Diamond, tokens per answer

      −68%

      Fast at 32,768 tokens, Qwen3.5-4B given 81,920 · Qwen's preset · accuracy level within noise (75.4% vs 75.9%) · lower is better

      Noema 1.5 4B Fast
      9,385
      Qwen3.5-4B
      28.9K
  3. Size
    2B
    Built on
    Qwen3.5-2B
    License
    Open weights
    Hugging Face (Noema 1.5 2B weights, opens in a new tab)

    A compact model with better instruction following and shorter reasoning.

    Selected results against Qwen3.5-2B. A one-time final evaluation of the frozen release against stock Qwen3.5-2B, paired on the same prompts. GPQA Diamond and IFBench finished below stock Qwen3.5-2B.

    Full results
    • IFEval

      +7.0 pts

      Instruction following · prompt strict, 541 prompts

      Noema 1.5 2B
      72.1%
      Qwen3.5-2B
      65.1%
    • Multi-IF

      +3.1 pts

      Multi-turn instructions · all three turns, 4,501 conversations

      Noema 1.5 2B
      17.9%
      Qwen3.5-2B
      14.8%
    • Thinking tokens

      −13.5%

      Mean completion, 300 prompts · accuracy 68.7% vs 67.3% · lower is better

      Noema 1.5 2B
      6,895
      Qwen3.5-2B
      7,969

Full results and methods are on each model page. Noema 1.5 models are open weights, not open source.

All models

Research

Our latest paper.

Research from NoemaAI and our collaborators, shared openly.

Type
Paper · Preprint
Published
Venue
Zenodo · Open access

Reusing Model Probabilities for New Queries

  • Armin StamateNoemaAI
  • Victoria Paraoulaki de MirandaHarvard University
  • Wyatt WunkerStanford University

Can one set of model probabilities answer many different questions? This paper studies how fine-grained predictions can be combined into broader categories, keeping answers consistent across label groupings without querying a model again for each grouping.

The Noema app

See what Noema does on your devices.

Noema runs language models on iPhone, iPad, Mac and Apple Vision Pro. Local models, documents and conversations can stay on your device. It is free on the App Store, and personal use does not require a Noema account.

Noema 1.5 4B Deep on Mac searching the web, reading three sources and answering when the James Webb Space Telescope launched

Search the web when a question needs it, then check the sources behind the answer. Shown with Noema 1.5 4B Deep on a Mac.

About Noema Search

Built to be trusted

Private by default, open by design.

  • Privacy

    Local by default

    Local models, documents and conversations can stay on your device. Every optional online feature is documented with where it connects, and Off-grid Mode blocks external HTTP and HTTPS traffic made through Noema.

    Network activity, feature by feature
    Where data goes, for selected features
    FeatureDestination
    Local models and retrievalYour device
    Python, memory and chartsYour device
    Web Search and Deep ResearchSelected search service and source reader
    Constellation chat syncYour private iCloud database
    Remote endpointsProvider you configure
  • Enterprise

    Noema for organizations

    Deploy Noema across a team with Noema Teams, answer from approved company documents, and pilot a model adapted to your work.

    Explore enterprise
  • Developers

    Build with Noema models

    Download the weights from Hugging Face, or serve models from Noema on your Mac to compatible clients through its Local API.

    • Transformers
    • GGUF
    • MLX
    Start building

Use our models in Noema or in your own stack.

Get Noema free on iPhone, iPad, Mac and Apple Vision Pro, or download open-weight Noema models to run on your own hardware.