Instructions to use MTSmash/EvaGPT-German-v9-1-2-4-7-latest with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MTSmash/EvaGPT-German-v9-1-2-4-7-latest with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MTSmash/EvaGPT-German-v9-1-2-4-7-latest")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MTSmash/EvaGPT-German-v9-1-2-4-7-latest")
model = AutoModelForCausalLM.from_pretrained("MTSmash/EvaGPT-German-v9-1-2-4-7-latest")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use MTSmash/EvaGPT-German-v9-1-2-4-7-latest with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MTSmash/EvaGPT-German-v9-1-2-4-7-latest"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MTSmash/EvaGPT-German-v9-1-2-4-7-latest",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MTSmash/EvaGPT-German-v9-1-2-4-7-latest

SGLang

How to use MTSmash/EvaGPT-German-v9-1-2-4-7-latest with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MTSmash/EvaGPT-German-v9-1-2-4-7-latest" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MTSmash/EvaGPT-German-v9-1-2-4-7-latest",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MTSmash/EvaGPT-German-v9-1-2-4-7-latest" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MTSmash/EvaGPT-German-v9-1-2-4-7-latest",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use MTSmash/EvaGPT-German-v9-1-2-4-7-latest with Docker Model Runner:
```
docker model run hf.co/MTSmash/EvaGPT-German-v9-1-2-4-7-latest
```

why old model

by kalle07 - opened 27 days ago

Discussion

kalle07

27 days ago

32k context?
not usuable most times
ministral-v3- 3b or 8b would be better as base ...

MTSmash

Owner 26 days ago

It's true that this is the v0.1 model, but we're only using the raw core, meaning the tokenizer. The model itself was completely re-initialized and trained from scratch. We're still working on improving the dataset so that the results get better. Since this is my community project, it unfortunately takes some time to find the right parts in the dataset to improve it. But we will also try out the other models, like the Minstral-v3 3B or 8B, in the future.

Let me know if you need it more concise or want to adjust the wording!

kalle07

about 4 hours ago

more or less a funny model ...

MTSmash

Owner about 3 hours ago

•

edited about 3 hours ago

The model was trained to work with additional sources from Google. If these are missing, the responses will be inaccurate. However, thank you for the feedback; this allows us to incorporate this aspect into our dataset as well.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment