Core concepts

Models

One entrypoint to any model from any company. Native adapters for Anthropic, OpenAI, and Gemini; everything else through the bundled LiteLLM universal adapter.

create_model

models.py
from antraft import create_model
 
create_model("anthropic:claude-opus-4-8")      # native
create_model("openai:gpt-4o")
create_model("gemini:gemini-2.5-flash")
create_model("groq:llama-3.1-70b", via="litellm")
create_model("bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0")   # AWS
create_model("azure:my-deployment", via="litellm")                  # Azure
create_model("ollama", model="llama3", base_url="http://localhost:11434/v1")
Note
Any OpenAI-compatible endpoint works via base_url — Ollama, vLLM, LM Studio, Together, OpenRouter, and others — with no code changes.

Caching

Wrap any model to cache identical requests. Cache hits report zero usage, so budgets stay accurate.

cache.py
from antraft.models import CachingModel
model = CachingModel(create_model("openai:gpt-4o"))
model.hit_rate   # 0.0 .. 1.0

Streaming

stream.py
model.stream(messages, on_token=lambda t: print(t, end=""))

Write your own adapter

Implement one method on ChatModel and your provider plugs into everything else.

custom_model.py
from antraft.models import ChatModel, ModelResponse, ToolCall
 
class MyModel(ChatModel):
    model = "my-model"
    def generate(self, messages, tools=None) -> ModelResponse:
        # call your API, then normalize:
        return ModelResponse(text="...", tool_calls=[], usage={"input_tokens": 0, "output_tokens": 0})

Cost tracking

Token usage is metered from each provider response; cost is computed from a pricing table. Models absent from the table report $0.0 for cost (token budgets still apply).

Next →
Policies