vllm — Easy Playground

High-throughput LLM serving engine with PagedAttention

Getting started with vllmRun locally

Install

pip install vllm

Python CodeRun locally

# Install: pip install vllm
import vllm

# Basic vllm usage
print(f"Using vllm")
# See documentation for detailed examples

# Install: pip install vllm
import vllm

# Basic vllm usage
print(f"Using vllm")
# See documentation for detailed examples

Expected Output

# Expected output shown below
# (Run locally with: vllm)

vllm is a third-party package. High-throughput LLM serving engine with PagedAttention. Install with: pip install vllm

Try modifying the code above to explore different behaviors. Can you extend the example to handle a new use case?