GadflyII/Qwen3-Coder-Next-NVFP4

Qwen

https://github.com/Gadflyii/vllm/tree/main NVFP4 quantized version of [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next) (80B-A3B). Quantized using [llmcompressor](https://github.com/vllm-project/llm-compressor) 0.9.0.1. NUM_CALIBRATION_SAMPLES = 20 MAX_SEQUENCE_LENGTH = 2048 DATASET = "HuggingFaceH4/ultrachat_200k" (train_sft) moe_calibrate_all_experts = True ignore = [ "lm_head", "re:.*mlp.gate$", # MoE router gates "re:.*mlp.shared_expert_gate$", # Shared expert gates "re:.*linear_attn.*", # DeltaNet linear attention ] Successfully tested up to **128K tokens** with FP8 KV cache (Not enough VRAM to test any higher context). Requires vLLM with NVFP4 support (0.16.0+), Transformers 5.0.0+ vllm serve GadflyII/Qwen3-Coder-Next-NVFP4 \ --tensor-parallel-size 2 \ --max-model-len 131072 \ --kv-cache-dtype fp8 Apache 2.0 (same as base model) - [Qwen Team](https://huggingface.co/Qwen) for the base model - [RedHatAI](https://huggingface.co/RedHatAI) for the quantization approach reference - [vLLM Project](https://github.com/vllm-project/vllm) for llmcompressor