vLLM

Pinned Loading

vllm vllm Public

A high-throughput and memory-efficient inference and serving engine for LLMs

Python 64.4k 11.7k
llm-compressor llm-compressor Public

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Python 2.3k 297
recipes recipes Public

Common recipes to run vLLM

Jupyter Notebook 250 96

Repositories

vllm-ascend Public
Community maintained hardware plugin for vLLM on Ascend

vllm-project/vllm-ascend’s past year of commit activity

Python 1,400 Apache-2.0 619 772 (8 issues need help) 260 Updated Dec 1, 2025
vllm Public
A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-project/vllm’s past year of commit activity

Python 64,371 Apache-2.0 11,681 1,901 (34 issues need help) 1,271 Updated Dec 2, 2025
tpu-inference Public
TPU inference for vLLM, with unified JAX and PyTorch support.

vllm-project/tpu-inference’s past year of commit activity

Python 171 Apache-2.0 46 17 (1 issue needs help) 65 Updated Dec 2, 2025
ci-infra Public
This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

vllm-project/ci-infra’s past year of commit activity

HCL 27 Apache-2.0 47 0 23 Updated Dec 1, 2025
vllm-gaudi Public
Community maintained hardware plugin for vLLM on Intel Gaudi

vllm-project/vllm-gaudi’s past year of commit activity

Python 18 Apache-2.0 74 1 69 Updated Dec 1, 2025
recipes Public
Common recipes to run vLLM

vllm-project/recipes’s past year of commit activity

Jupyter Notebook 250 Apache-2.0 95 7 7 Updated Dec 1, 2025
aibrix Public
Cost-efficient and pluggable Infrastructure components for GenAI inference

vllm-project/aibrix’s past year of commit activity

Go 4,442 Apache-2.0 490 251 (19 issues need help) 23 Updated Dec 1, 2025
vllm-spyre Public
Community maintained hardware plugin for vLLM on Spyre

vllm-project/vllm-spyre’s past year of commit activity

Python 37 Apache-2.0 29 4 16 Updated Dec 1, 2025
llm-compressor Public
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vllm-project/llm-compressor’s past year of commit activity

Python 2,297 Apache-2.0 296 80 (15 issues need help) 46 Updated Dec 1, 2025
semantic-router Public
Intelligent Router for Mixture-of-Models

vllm-project/semantic-router’s past year of commit activity

Go 2,347 Apache-2.0 301 106 (24 issues need help) 35 Updated Dec 1, 2025

View all repositories

Sponsors

View all

Top languages

Python C++ Go Cuda Jupyter Notebook

Most used topics

inference llm model-serving transformer pytorch