Use Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "digitous/Javalion-R" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "digitous/Javalion-R",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Javalion-R is a penta merge of KoboldAI's GPT-J classics + PygmalionAI's Pygmalion6b;
((Janeway + Shinen) + (Skein + Pygmalion)) + GPT-R.
Janeway + Shinen is listed under JANIN-GPTJ. Skein + Pygmalion is listed under SKEGMA-GPTJ.
GPT-R itself is a 60/40 merge of two instruct research models (see digitous/GPT-R for full credits).
This 5x+ merge is not intended for minors, as it can produce NC-17+ content.
This model differs from Javelin-R by substituting the Adventure model with Pygmalion, as Adventure is rendered redundant in training data by Skein. Javalion-R is a research artefact with dual purpose for entertainment as well as an intended example of potential value instruct can bring when combined with models of a different purpose through the use of weight sum merge technology.
Mileage mat vary. No refunds best wishes. Mainly intended to be utilized with Open Source KoboldAI software. Optimal sampler and settings not determined. Feedback Welcome!
- Downloads last month
- 961
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "digitous/Javalion-R" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "digitous/Javalion-R", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'