Skip to main content
This page shows how to:
  • Use a unified API. TensorZero provides the same chat completion format for the Responses API.
  • Access built-in tools. Enable built-in tools from OpenAI like web_search.
  • Enable reasoning models. Support models with extended thinking capabilities.
You can find a complete runnable example of this guide on GitHub.

​
Call the OpenAI Responses API

You can point the OpenAI Python SDK to a TensorZero Gateway to access the Responses API.
1

Set up your OpenAI API key

You can set the OPENAI_API_KEY environment variable with your API key.
export OPENAI_API_KEY="sk-..."
2

Install the OpenAI Python SDK

You can install the OpenAI SDK with a Python package manager like pip.
pip install openai
3

Configure a model for the OpenAI Responses API

Create a configuration file with a model using api_type = "responses" and provider tools:
tensorzero.toml
[models.gpt-5-mini-responses-web-search]
routing = ["openai"]

[models.gpt-5-mini-responses-web-search.providers.openai]
type = "openai"
model_name = "gpt-5-mini"
api_type = "responses"
include_encrypted_reasoning = true
provider_tools = [{type = "web_search"}]  # built-in OpenAI web search tool
# Enable plain-text summaries of encrypted reasoning
extra_body = [
    { pointer = "/reasoning", value = { effort = "low", summary = "auto" } }
]
4

Deploy the TensorZero Gateway

Let’s deploy the TensorZero Gateway using Docker. For simplicity, we’ll use the gateway with the configuration above.
docker run \
  -e OPENAI_API_KEY \
  -v $(pwd)/tensorzero.toml:/app/config/tensorzero.toml:ro \
  -p 3000:3000 \
  tensorzero/gateway \
  --config-file /app/config/tensorzero.toml
See the Deploy the TensorZero Gateway page for more details.
5

Initialize the OpenAI client

Let’s initialize the OpenAI SDK and point it to the gateway we just launched.
from openai import OpenAI

client = OpenAI(base_url="http://localhost:3000/openai/v1", api_key="not-used")
6

Call the LLM

OpenAI web search can take up to a minute to complete.
response = client.chat.completions.create(
    model="tensorzero::model_name::gpt-5-mini-responses-web-search",
    messages=[
        {
            "role": "user",
            "content": "What is the current population of Japan?",
        }
    ],
)

ChatCompletion(
    id='0199ff78-5bad-7312-ab13-e4c5fa0bde8d',
    choices=[
        Choice(
            finish_reason='stop',
            index=0,
            logprobs=None,
            message=ChatCompletionMessage(
                content="Short answer — it depends on the source/date:\n\n- Japan's official demographic survey (Ministry of Internal Affairs and Communications, reported by major Japanese outlets) shows a total population of 124,330,690 as of January 1, 2025 (this includes foreign residents). ([asahi.com](https://www.asahi.com/ajw/articles/15952384?utm_source=openai))\n\n- International mid‑year estimates (United Nations/UNFPA) put Japan's 2025 population at about 123.1 million (mid‑2025 estimate), which uses a different methodology and reference date. ([unfpa.org](https://www.unfpa.org/data/world-population/JP?utm_source=openai))\n\nToday is October 20, 2025 — would you like me to fetch a live or another specific estimate (e.g., UN mid‑year, World Bank, or the latest Japanese government update)?",
                refusal=None,
                role='assistant',
                annotations=None,
                audio=None,
                function_call=None,
                tool_calls=[]
            )
        )
    ],
    created=1760927745,
    model='tensorzero::model_name::gpt-5-mini-responses-web-search',
    object='chat.completion',
    service_tier=None,
    system_fingerprint='',
    usage=CompletionUsage(
        completion_tokens=2304,
        prompt_tokens=21444,
        total_tokens=23748,
        completion_tokens_details=None,
        prompt_tokens_details=None
    ),
    episode_id='0199ff78-5bad-7312-ab13-e4d8708e5b73'
)

​
Call the OpenAI Responses API with Azure

You can call the OpenAI Responses API with Azure by setting api_base in your configuration to your Azure deployment URL. 
[models.azure-gpt-5-mini-responses]
routing = ["azure"]

[models.azure-gpt-5-mini-responses.providers.azure]
type = "openai"  # CAREFUL: not `azure`!
api_base = "https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/"  # TODO: Insert your API base URL here
api_key_location = "env::AZURE_API_KEY"
model_name = "gpt-5-mini"
api_type = "responses"
The azure model provider does not support the Responses API. You must use the openai provider with a custom api_base instead.