Cost Optimizer

Automatically route requests to cheaper models when possible. Define rules based on priority, complexity, and budget to maximize savings without sacrificing quality.

What is the Cost Optimizer?

The Cost Optimizer is an intelligent routing engine that directs AI requests to the most cost-effective model based on rules you define. For example, simple FAQ questions can go to GPT-4o-mini ($0.15/1M tokens) instead of GPT-4o ($2.50/1M tokens) — saving up to 94%.

Create an Optimization Rule

Create Cost Rulebash

curl -X POST http://localhost:8000/api/v1/cost/rules \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "name": "route-simple-to-mini",
    "description": "Route simple queries to cheaper models",
    "condition_type": "priority",
    "condition_value": "low",
    "target_model": "gpt-4o-mini",
    "fallback_model": "gpt-4o",
    "enabled": true
  }'

Track Savings

Track Savingsbash

# View cumulative savings
curl http://localhost:8000/api/v1/cost/savings \
  -H "Authorization: Bearer YOUR_API_KEY"

# Get cost estimates for different models
curl -X POST http://localhost:8000/api/v1/cost/estimate \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{ "prompt_tokens": 500, "completion_tokens": 200 }'

Quick Win

Start by routing all "low" priority requests to a mini model. Most users see 40-60% cost savings with no noticeable quality loss for simple tasks.

Marketplace

Providers