Instructions to use rpDungeon/Gemma4-31b-Gembrain-Equinox with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rpDungeon/Gemma4-31b-Gembrain-Equinox with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="rpDungeon/Gemma4-31b-Gembrain-Equinox")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("rpDungeon/Gemma4-31b-Gembrain-Equinox")
model = AutoModelForImageTextToText.from_pretrained("rpDungeon/Gemma4-31b-Gembrain-Equinox")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use rpDungeon/Gemma4-31b-Gembrain-Equinox with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rpDungeon/Gemma4-31b-Gembrain-Equinox"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rpDungeon/Gemma4-31b-Gembrain-Equinox",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/rpDungeon/Gemma4-31b-Gembrain-Equinox

SGLang

How to use rpDungeon/Gemma4-31b-Gembrain-Equinox with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rpDungeon/Gemma4-31b-Gembrain-Equinox" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rpDungeon/Gemma4-31b-Gembrain-Equinox",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rpDungeon/Gemma4-31b-Gembrain-Equinox" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rpDungeon/Gemma4-31b-Gembrain-Equinox",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use rpDungeon/Gemma4-31b-Gembrain-Equinox with Docker Model Runner:
```
docker model run hf.co/rpDungeon/Gemma4-31b-Gembrain-Equinox
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Gemma4-31b-Gembrain-Equinox

A V2 Fisher-protected community merge of two Gemma-4 31B creative-writing variants on top of stock google/gemma-4-31b-it. Built for the jaxxks / twisted / toasty research thread on instruct-preserving merges.

What's inside

Source	Role in the merge
`google/gemma-4-31b-it`	base / instruct anchor
Gembrain (community 31B)	style blender — reinforces prompt-attention
Equinox (community 31B)	unique style, neutral/realistic — best at low percentages

Recipe sketch: TIES-style merge of the two community deltas (Gembrain − IT, Equinox − IT) into IT, with Fisher importance + layer-importance damping applied to the combined delta to protect the high-importance instruct-following parameters. Scale 0.3.

Per jaxxks's characterization (2026-05-23): MeroMero would be more attention-to-detail-leaning; Gembrain reinforces user-prompt attention; Equinox carries a neutral/realistic finishing edge but is best at low percentages — the Fisher+layer damping is what lets us include Equinox at a non-trivial weight without losing instruct adherence.

Chat template — important

This repo ships a modified chat_template.jinja that prefills <|channel>thought\n on assistant turns when add_generation_prompt=True and enable_thinking=True. This patches a previously-observed "merges skip thinking" regression where the assistant would open a generation without entering the thought channel first.

enable_thinking=True → assistant turn begins inside the thought channel; produces a thought trace then folds back to the answer.
enable_thinking=False → behaves identically to stock Gemma-4 (empty thought-block convention preserved).

If you're loading via transformers, you'll pick this up automatically because chat_template.jinja is in the repo root. If you're loading via llama.cpp from a re-quantized GGUF, make sure the GGUF was built after the chat template was placed in the source dir, or apply it post-hoc via gguf_new_metadata.py --chat-template-file.

GGUF

A Q4_K_M GGUF of this exact merge (with the chat-template fix) is mirrored at:

ToastyPigeon/Gemma4-Test-GGUFs/g4_31b_v2_gembrain_equinox.Q4_K_M.gguf

Recommended llama-server flags for Gemma 4 31B at Q4_K_M on a dual-3090 box (subject to verification — see EVAL_GUIDELINES):

CUDA_VISIBLE_DEVICES=0,1 llama-server \
  --model g4_31b_v2_gembrain_equinox.Q4_K_M.gguf \
  --tensor-split 1,1 \
  --ctx-size 8192 --n-gpu-layers 999 \
  --parallel 8 --no-warmup --no-mmap --jinja \
  -fa on -ctk q8_0 -ctv q8_0

Client requests: cache_prompt: true. Do not pass --swa-full (default SWA-on is correct for Gemma 4).

Known caveats

This is a V2 merge. The V3 fisher+layer family (ToastyPigeon/g4-31b-v3-fisher-layer-test) supersedes it in our internal release thread but V2 remains an interesting "low-scale community blend on IT" data point.
Eval-side regression vs the local 31B-IT baseline was measured at roughly -19.78pp IFEval-strict in our first end-to-end test (2026-05-23). Most of that gap traced back to a chat-template-missing bug at the GGUF level (now fixed); a re-eval against the template-fixed GGUF is on the post-eval-opt queue.
The two source community models have their own licensing terms; check both before redistribution.

Reproduction notes (high level)

Pull stock google/gemma-4-31b-it weights.
Pull the two source community 31B variants (Gembrain, Equinox).
Compute Δ_GB = W_GB − W_IT, Δ_EQ = W_EQ − W_IT.
TIES-merge the two deltas (sign-conflict resolution + magnitude pruning).
Apply a Fisher + layer-importance mask to damp instruct-critical parameters, then add to IT at scale 0.3.
Copy chat_template.jinja (with the thought-channel prefill) into the merged dir before any quant step.
Save merged bf16 safetensors. Optionally convert to GGUF + quant.

For the masking technique itself, see INSTRUCT_MASKING_TECHNIQUES.md (cross-reference) or the project's instruct_mask/INSTRUCT_MASKING_TECHNIQUES.md.

Credits

Merge recipe: jaxxks (characterization) + twisted (mask infra) + toasty (training / orchestration)
Stock base: Google's Gemma 4 31B IT
Community variants: Gembrain, Equinox (originating authors retain credit for the source models)

Downloads last month: 42

Safetensors

Model size

33B params

Tensor type

BF16

Model tree for rpDungeon/Gemma4-31b-Gembrain-Equinox

Quantizations

3 models

Collection including rpDungeon/Gemma4-31b-Gembrain-Equinox

Models

Collection

Some stuff that's kinda usable • 2 items • Updated 1 day ago