Baidu: ERNIE 4.5 21B A3B

A sophisticated text-based Mixture-of-Experts (MoE) model featuring 21B total parameters with 3B activated per token, delivering exceptional multimodal understanding and generation through heterogeneous MoE structures and modality-isolated routing. Supporting an extensive 131K token context length, the model achieves efficient inference via multi-expert parallel collaboration and quantization, while advanced post-training techniques including SFT, DPO, and UPO ensure optimized performance across diverse applications with specialized routing and balancing losses for superior task handling.

Other versions in the ERNIE family

Baidu: ERNIE 4.5 21B A3B Thinking

21B

Baidu: ERNIE 4.5 300B A47B

300B

Baidu: ERNIE 4.5 VL 28B A3B

28B

Baidu: ERNIE 4.5 VL 424B A47B

424B