Bubbles-CLI-Qwen3.5-35B-A3B-LoRA

QLoRA adapter fine-tuned on CLI-Anything style workflows for AI/ML, RL, and Data Analytics tool orchestration. Teaches the model to generate structured CLI commands, chain tools in pipelines, parse JSON output with jq, and handle errors with corrected retries.

Built for the ablac Arena bot project.

Training

Parameter	Value
Base Model	huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated
Method	QLoRA (4-bit NF4, double quant)
LoRA Rank / Alpha	64 / 128
Target Modules	q, k, v, o, gate, up, down proj
Examples	500 (450 train / 50 eval)
Epochs	5
Steps	285
Train Loss	0.304
Eval Loss	0.236
Token Accuracy	92.1%
Time	58.6 min on 2x H200
Framework	transformers 5.3.0-dev, peft 0.18.1, trl 0.29.0

What It Does

• Single-tool workflows (200 examples) - PyTorch, scikit-learn, XGBoost, Gymnasium, DuckDB, MLflow, Matplotlib + 22 more
• Multi-tool pipelines (150 examples) - DuckDB prep, sklearn train, MLflow log, Plotly viz
• Error handling (100 examples) - wrong flags, missing deps, CUDA OOM, with diagnosis + retry
• CLI-Anything synthesis (50 examples) - ca repo scan and ca cli synth workflows

Example

Prompt: Train a random forest on customer data and evaluate it.

Response:
skfit train --input train.parquet --target label --model random_forest \
  --n-estimators 200 --max-depth 10 --test-size 0.2 \
  --metrics roc_auc,accuracy,f1 --output-dir ./artifacts/rf_model \
  --json > train_result.json

jq '{roc_auc, training_time_s, f1}' train_result.json

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated",
    device_map="auto", torch_dtype="auto",
)
model = PeftModel.from_pretrained(base, "ademczuk/Bubbles-CLI-Qwen3.5-35B-A3B-LoRA")
tokenizer = AutoTokenizer.from_pretrained("ademczuk/Bubbles-CLI-Qwen3.5-35B-A3B-LoRA")

Tools Covered

29 tools: pytorch, sklearn, xgboost, lightgbm, huggingface, jax, tensorflow, gymnasium, sb3, rllib, cleanrl, duckdb, polars, arrow, pandas, spark, matplotlib, plotly, seaborn, altair, mlflow, wandb, dvc, airflow, bentoml, torchserve, triton, fastapi, cli_anything.

Loss Curve

Step   5: loss=1.813  acc=64%
Step  50: loss=0.386  acc=88%
Step 100: loss=0.230  acc=92%  eval=0.236
Step 200: loss=0.160  acc=93%
Step 285: loss=0.304  acc=92%  (cosine decay)

Framework versions

• PEFT 0.18.1
• Transformers 5.3.0.dev0
• TRL 0.29.0
• PyTorch 2.5.1+cu124