Skip to main content
This page shows how to:
  • Use a unified API. TensorZero provides the same chat completion format for the Responses API.
  • Access built-in tools. Enable built-in tools from OpenAI like web_search.
  • Enable reasoning models. Support models with extended thinking capabilities.
We provide complete code examples on GitHub.

​
Call the OpenAI Responses API

  • Python (TensorZero SDK)
  • Python (OpenAI SDK)
  • Node (OpenAI SDK)
  • HTTP
The TensorZero Python SDK provides a unified API for calling OpenAI’s Responses API.
1

Set up your OpenAI API key

You can set the OPENAI_API_KEY environment variable with your API key.
export OPENAI_API_KEY="sk-..."
2

Install the TensorZero Python SDK

You can install the TensorZero SDK with a Python package manager like pip.
pip install tensorzero
3

Configure a model for the OpenAI Responses API

Create a configuration file with a model using api_type = "responses" and provider tools:
tensorzero.toml
[models.gpt-5-mini-responses-web-search]
routing = ["openai"]

[models.gpt-5-mini-responses-web-search.providers.openai]
type = "openai"
model_name = "gpt-5-mini"
api_type = "responses"
include_encrypted_reasoning = true
provider_tools = [{type = "web_search"}]  # built-in OpenAI web search tool
4

Deploy a standalone (HTTP) TensorZero Gateway

Let’s deploy a standalone TensorZero Gateway using Docker. For simplicity, we’ll use the gateway with the configuration above.
docker run \
  -e OPENAI_API_KEY \
  -v $(pwd)/tensorzero.toml:/app/config/tensorzero.toml:ro \
  -p 3000:3000 \
  tensorzero/gateway \
  --config-file /app/config/tensorzero.toml
See the TensorZero Gateway Deployment page for more details.
5

Initialize the TensorZero Gateway client

Let’s initialize the TensorZero Gateway client and point it to the gateway we just launched.
from tensorzero import TensorZeroGateway

t0 = TensorZeroGateway.build_http(gateway_url="http://localhost:3000")
The TensorZero Python SDK includes a synchronous TensorZeroGateway client and an asynchronous AsyncTensorZeroGateway client. Both options support running the gateway embedded in your application with build_embedded or connecting to a standalone gateway with build_http. See Clients for more details.
6

Call the LLM

OpenAI web search can take up to a minute to complete.
response = t0.inference(
    model_name="gpt-5-mini-responses-web-search",
    input={
        "messages": [
            {
                "role": "user",
                "content": "What is the current population of Japan?",
            }
        ]
    },
)

ChatInferenceResponse(
    inference_id=UUID('0199ff78-6246-7c12-b4b0-6e3a881cc6b9'),
    episode_id=UUID('0199ff78-6246-7c12-b4b0-6e4367f949b8'),
    variant_name='gpt-5-mini-responses-web-search',
    content=[
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo9...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fda57d481969c3603df0d675348',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': 'Japan population 2025 October 2025 population estimate Statistics Bureau of Japan'
                }
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fdf9b988196b36756d639e2b015',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "Ministry of Internal Affairs and Communications Japan population Oct 1 2024 'total population' 'Japan' 'population estimates' '2024' 'Oct. 1' '総人口' '令和6年' "
                }
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe1a388819684971acfdaf4cd44',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "Ministry of Internal Affairs and Communications population Japan Oct 1 2024 total population 'Oct. 1, 2024' 'population' 'Japan' 'MIC' 'population estimates' '2024' '総人口' "
                }
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe439788196911a195c70cc8ca9',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {'type': 'search'}
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe6b140819690a4468d3304fece',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {'type': 'search'}
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59fe81e408196921b69174f6abaf7',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {'type': 'search'}
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59feda6188196827a0b5aa01e96a1',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "United Nations World Population Prospects 2024 Japan 2025 population 'Japan population 2025' 'World Population Prospects 2024' 'Japan' "
                }
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59ff3cc8881968d1c5c9c1bbe4ecc',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "UN World Population Prospects 2024 Japan population 2025 '123,103,479' 'Japan 2025' 'World Population Prospects' 'Japan' '2025' "
                }
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        UnknownContentBlock(
            data={
                'id': 'ws_05489a0b57dc84980168f59ff67ed48196a0054a38e96f8e0c',
                'type': 'web_search_call',
                'status': 'completed',
                'action': {
                    'type': 'search',
                    'query': "United Nations population Japan 2025 'World Population Prospects 2024' 'Japan population 2025' site:un.org"
                }
            },
            model_provider_name='tensorzero::model_name::gpt-5-mini-responses-web-search::provider_name::openai',
            type='unknown'
        ),
        Thought(
            text=None,
            type='thought',
            signature='gAAAAABo...',
            _internal_provider_type=None
        ),
        Text(
            text="Short answer: about 123–124 million people.\n\nMore precisely:\n- Japan's official estimate (Ministry of Internal Affairs and Communications / e‑Stat) reported a total population of 123,802,000 (including foreign residents) as of October 1, 2024 (release published Apr 14, 2025). ([e-stat.go.jp](https://www.e-stat.go.jp/en/stat-search/files?layout=dataset&page=1&query=Population+Estimates%2C+natural))  \n- The United Nations (WPP 2024, used by sources such as Worldometer) gives a mid‑2025 estimate of about 123.1 million. ([srv1.worldometers.info](https://srv1.worldometers.info/world-population/japan-population/?utm_source=openai))\n\nDo you want a live "right now" estimate for today (Oct 20, 2025) or a breakdown by Japanese nationals vs. foreign residents? I can fetch the latest live or official figures for the exact date you want.",
            arguments=None,
            type='text'
        )
    ],
    usage=Usage(input_tokens=29904, output_tokens=1921),
    finish_reason=None,
    original_response=None
)
⌘I