---
title: "Prompt format and sampling"
description: "How Noema models format conversations, how to turn thinking on and off, and the sampling and output limits each model was evaluated with."
version: "Noema 1.5 · Noema 2B"
applies_to: "Transformers · llama.cpp · MLX · vLLM"
reviewed: "2026-10-23"
canonical: "https://noemaai.com/docs/prompt-format"
---

# Prompt format and sampling

Chat template, thinking and non-thinking modes, and the sampling and output limits each model card recommends.

## Use the embedded chat template

Every repository ships the model's chat template: `chat_template.jinja` beside the Transformers and MLX weights, and embedded in each GGUF file. Let your runtime apply it instead of writing prompt markup by hand: `apply_chat_template` in Transformers, `--jinja` in llama.cpp, and the chat endpoints of llama.cpp and vLLM.

- End-of-sequence token: `<|im_end|>`. Padding token: `<|endoftext|>`.
- Text in, text out. The base model's vision encoder is not included.

## Thinking and non-thinking

Noema 1.5 4B thinks by default and switches through the chat template. Noema 1.5 2B is recommended without thinking for instructions, structured output and code. Noema 2B was trained and evaluated without thinking.

| Model | Modes | Context |
| --- | --- | --- |
| [Noema 1.5 4B](https://noemaai.com/models/noema-1-5-4b) | Thinking (default) · non-thinking, selected through the chat template | 262,144 tokens (architecture limit); outputs evaluated to 81,920 |
| [Noema 1.5 2B](https://noemaai.com/models/noema-1-5-2b) | Non-thinking (recommended) · thinking | 262,144 tokens (architecture limit); independently validated to 24,576 |
| [Noema 2B](https://noemaai.com/models/noema-2b) | Non-thinking (trained and evaluated); thinking mode is not supported | 262,144 tokens (architecture limit) |

**Transformers**

```python
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
)
```

**llama.cpp**

```bash
llama-server -hf NoemaAI-labs/Noema-1.5-4B-Fast-GGUF:Q4_K_M --jinja \
  --chat-template-kwargs '{"enable_thinking": false}'
```

**MLX**

```bash
mlx_lm.generate --model NoemaAI-labs/Noema-1.5-4B-Fast-MLX-4bit \
  --prompt "Summarize this paragraph." \
  --chat-template-config '{"enable_thinking":false}'
```

**vLLM (OpenAI client)**

```python
response = client.chat.completions.create(
    model="NoemaAI-labs/Noema-1.5-4B-Fast",
    messages=messages,
    temperature=0.7,
    top_p=0.8,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": False}},
)
```

## Read the answer

With thinking on, the reasoning trace comes first and ends at `</think>`; the answer follows. Split on the last `</think>`, or let vLLM's `qwen3` reasoning parser return the two separately.

```python
reasoning, _, answer = text.rpartition("</think>")
print(answer.strip())
```

In multi-turn use, keep only the final answers in the conversation history. The Noema 1.5 4B chat template already drops earlier reasoning traces. In the Noema app, [Reasoning history](https://noemaai.com/docs/reasoning-history) controls this for supported GGUF models.

## Sampling for Noema 1.5 4B

| Use | Thinking | Temperature | Top-p | Top-k | Presence penalty | Output limit |
| --- | --- | --- | --- | --- | --- | --- |
| General reasoning (recommended default) | On | 0.6 | 0.95 | 20 | 0 | 32,768 |
| Competition math and programming (Deep only) | On | 0.6 | 0.95 | 20 | 0 | 81,920 |
| Qwen's general thinking preset | On | 1.0 | 0.95 | 20 | 1.5 | 32,768 |
| Direct answers, chat and structured output | Off | 0.7 | 0.8 | 20 | 1.5 | 8,192 to 16,384 |

> Thinking is on by default; pass enable_thinking=False to the chat template to answer directly. Avoid greedy decoding with thinking on, which makes repetition more likely in any reasoning model. In multi-turn use, keep only final answers in the history.

## Sampling for Noema 1.5 2B

| Use | Thinking | Temperature | Top-p | Top-k | Presence penalty | Output limit |
| --- | --- | --- | --- | --- | --- | --- |
| Instructions, structured output and code (recommended) | Off | 0 (greedy) | 1 | Off | 0 | Set a limit |
| Harder reasoning | On | 1.0 | 0.95 | 20 | 1.5 | Set a limit |

> The packaged defaults are greedy and stop at <|im_end|>. Deployments should enforce output limits and detect repeated spans, especially with thinking on.

## Sampling for Noema 2B

| Use | Thinking | Temperature | Top-p | Top-k | Presence penalty | Output limit |
| --- | --- | --- | --- | --- | --- | --- |
| Non-thinking (trained and evaluated) | Off | 0.7 | 0.8 | 20 | 1.5 | Set a limit |

> Do not use greedy decoding: this model family can enter repetition loops under greedy sampling. The presence penalty applies on engines that support it, such as vLLM and llama.cpp.

## Related documentation
- [Run Noema models](https://noemaai.com/docs/run-noema-models)
- [Noema models](https://noemaai.com/docs/models)
- [Evaluation methodology](https://noemaai.com/docs/evaluation-methodology)
- [Reasoning history](https://noemaai.com/docs/reasoning-history)