Intel/Qwen3.5-122B-A10B-gguf-q2ks-mixed-AutoRound

Model Details

This model is a mixed gguf q2ks format of Qwen/Qwen3.5-122B-A10B generated by intel/auto-round algorithm. Embedding layer and lm-head layer are fallback to 8 bits and non expert layers are fallback to 4 bits. Please refer to Section Generate the model for more details.

Please follow the license of the original model.

How To Use

Llama.cpp inference.

> wget https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg
> ./llama-cli  -m Qwen3.5-122B-A10B-w2g16/Qwen3.5-122B-A10B-Q2_K_S.gguf  -mm Qwen3.5-122B-A10B-w2g16/mmproj-model.gguf

> /image demo.jpeg

Loaded media from 'demo.jpeg'

> Describe this image.

[Start thinking]

The user wants a description of the image.

1.  **Identify the main subjects:** A woman and a dog.
2.  **Identify the setting:** A sandy beach with the ocean in the background. It looks like late afternoon or sunset due to the warm lighting.
3.  **Describe the action:** They are interacting. The dog is sitting and giving its paw to the woman. The woman is sitting on the sand, smiling, and holding the dog's paw. It looks like a "high five" or a handshake.
4.  **Describe the dog:** It's a light-colored dog, possibly a yellow Labrador or Golden Retriever. It's wearing a harness with colorful patterns. It's sitting on its hind legs.
5.  **Describe the woman:** She has long dark hair. She's wearing a plaid shirt (flannel) and dark pants or leggings. She is barefoot. She is smiling at the dog.
6.  **Describe the lighting/atmosphere:** The sun is low, creating a lens flare and a golden glow. It's a peaceful, happy, and warm atmosphere.
7.  **Describe the background:** The ocean waves are visible in the distance, slightly out of focus. The sand has footprints and texture.
8.  **Synthesize into a coherent description:** Start with the main action, then detail the subjects, then the setting and mood.

*Drafting the description:*
The image shows a heartwarming scene on a beach at what appears to be sunset. A young woman and a large, light-colored dog (likely a Labrador or Golden Retriever) are sitting on the sand facing each other.

*   **The Action:** The dog is sitting up on its hind legs and offering its front right paw to the woman. The woman is reaching out and holding the dog's paw, almost like a handshake or a high-five. She is smiling warmly at the dog.
*   **The Dog:** The dog is tan/yellow in color. It is wearing a blue harness with colorful paw prints on it.
*   **The Woman:** She has long dark hair and is wearing a plaid flannel shirt and dark pants. She is barefoot. She looks happy and relaxed.
*   **The Setting:** They are on a sandy beach. The ocean is in the background with small waves rolling in. The lighting is very bright and warm, suggesting the "golden hour" before sunset, with the sun creating a soft, hazy glow on the right side of the image.

*Refining:* Make it flow better. "A woman and her dog are enjoying a moment on the beach." "The dog is a yellow Labrador." "The sun is setting."

*Final Polish:* Ensure all key details are covered.
- Woman sitting cross-legged/legs extended.
- Dog sitting.
- Harness detail.
- Beach texture.
- Ocean waves.
- Lighting (backlit/sun flare).

This looks complete. I will structure the response to cover these points clearly.

[End thinking]

This image captures a heartwarming moment between a young woman and her dog on a sandy beach, likely during the "golden hour" of sunset.

**The Subjects:**
*   **The Woman:** She is sitting on the sand, facing left. She has long dark hair and is wearing a plaid flannel shirt and dark pants. She is barefoot and smiling warmly at the dog.
*   **The Dog:** It appears to be a yellow Labrador or similar breed. The dog is sitting on its hind legs and is wearing a blue harness with a colorful paw-print pattern.

**The Action:**
The two are interacting affectionately. The dog has lifted its front right paw, and the woman is holding it, creating a gesture that looks like a handshake or a high-five.

**The Setting:**
*   **Foreground:** The sand is textured with footprints and uneven surfaces.
*   **Background:** The ocean stretches out behind them with gentle waves rolling in.
*   **Lighting:** The scene is bathed in bright, warm sunlight c

Generate the model

Here is the sample command to reproduce the model

auto_round --model Qwen/Qwen3.5-122B-A10B --iters 0 --output_dir tmp_autoround --scheme gguf:q2_k_mixed

Ethical Considerations and Limitations

The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.

Therefore, before deploying any applications of the model, developers should perform safety testing.

Caveats and Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.

Here are a couple of useful links to learn more about Intel's AI software:

• Intel Neural Compressor link

Disclaimer

The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.

Cite

@article{cheng2023optimize, title={Optimize weight rounding via signed gradient descent for the quantization of llms}, author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi}, journal={arXiv preprint arXiv:2309.05516}, year={2023} }

arxiv github