Qwen3.8-27B-NVFP4 1M context. So far so good.

I am a beginner, Took a while to get started, get everything right.

This setup is native not container. Still not sure if I did this right, or if I can tune this more.

Environment=HF_HUB_OFFLINE=1 Environment=VLLM_LOGGING_LEVEL=INFO Environment=VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 Environment=PATH=/home/suryakiranc/vllm/.venv/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin Environment=CUDA_HOME=/usr/local/cuda ExecStart=/home/suryakiranc/vllm/.venv/bin/vllm serve unsloth/Qwen3.8-27B-NVFP4 \ --served-model-name unsloth/Qwen3.8-27B-NVFP4 \ --safetensors_load_strategy prefetch \ --tensor-parallel-size 4 \ --reasoning-parser qwen3 \ --tool-call-parser qwen3_xml \ --enable-auto-tool-choice \ --gpu-memory-utilization 0.91 \ --kv-cache-dtype fp8 \ --max-num-batched-tokens 16384 \ --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \ --mm-encoder-tp-mode data \ --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' \ --max-model-len 1000000 \ --host 0.0.0.0 \ --port 8000