RepoRadar
Leave a Tip
Item detail

lightseekorg/smg

SMG is an Apache-2.0, Rust-based engine-agnostic LLM gateway that exposes a single OpenAI- and Anthropic-compatible API across vLLM, TensorRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini, and similar backends. It adds a small routing and observability layer on top of the engines themselves without forcing a runtime change.

Open source →Back to radar
Score7.5
Popularity65.0
Risknone
TierGold
Score breakdown
Usefulness7.0
Novelty7.0
Momentum7.0
Maturity7.3
Open-source/build8.4
Evidence7.2
Workflow potential7.9
Setup ease6.4

Popularity is tracked separately. Support, ads, sponsorships, and tips never affect these signals.

Why it matters

Useful for teams running mixed inference backends who want a single client surface and routing layer: point SMG at your existing vLLM and OpenAI endpoints, swap the client base URL, and check that latency and cost dashboards still look right before tearing out the bespoke proxy.

Who should use it

ML platform teamsinference SREsteams running mixed local + cloud backendscost-routing experiments

Who should skip it

Skip if the source link, docs, or setup requirements do not match your workflow.

Risk explanation

gateways become a single point of failure; run with health checks and a fallback plan for direct backend access.

Evidence links

Closest alternatives / related signals

llm-gatewayrustvllmtrtllminferenceroutingopenai-compatible