Hugging Face Inference Providers: First API Request

Inference Providers is Hugging Face’s serverless API: one HF token, one endpoint, and each request is routed to a partner (Groq, Together, Cerebras, Fireworks and others) that runs the open model you named. No GPU, no download.

1. Token

Create a fine-grained token at huggingface.co/settings/tokens with the Make calls to Inference Providers permission, then export it:

export HF_TOKEN="hf_your_token_here"

$env:HF_TOKEN = "hf_your_token_here"

Both SDKs below read it from the environment. hf auth login stores a token on disk instead; HF_TOKEN wins when both exist.

2. InferenceClient (Python)

pip install -U huggingface_hub

from huggingface_hub import InferenceClient

client = InferenceClient()  # reads HF_TOKEN or the token saved by `hf auth login`

completion = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Explain HTTP 402 in one sentence."}],
)

print(completion.choices[0].message.content)

Sends one chat turn and prints the reply. chat.completions.create is an alias of chat_completion; both return the OpenAI response shape.

3. OpenAI SDK

pip install -U openai

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
)

completion = client.chat.completions.create(
    model="openai/gpt-oss-120b:fastest",
    messages=[{"role": "user", "content": "Explain HTTP 402 in one sentence."}],
)

print(completion.choices[0].message.content)

The same call through the OpenAI-compatible router: existing OpenAI code only needs a new base URL, your HF token and a Hub model id. This endpoint is chat-only; images, embeddings and speech go through InferenceClient.

4. curl

curl https://router.huggingface.co/v1/chat/completions \
  -H "Authorization: Bearer $HF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b:fastest",
    "messages": [{"role": "user", "content": "Explain HTTP 402 in one sentence."}],
    "stream": false
  }'

Returns one JSON chat completion. For token-by-token output see streaming responses. GET https://router.huggingface.co/v1/models lists every routed model with per-provider price, context length and throughput.

5. Provider and billing

By default the router picks the fastest live provider (highest tokens per second). A suffix on the model id changes that:

openai/gpt-oss-120b:fastest     # default policy
openai/gpt-oss-120b:cheapest    # lowest price per output token
openai/gpt-oss-120b:preferred   # your order in Inference Provider settings
openai/gpt-oss-120b:groq        # always this provider

With InferenceClient you can pass provider="groq" instead; the default is "auto", which also fails over when a provider is flagged unavailable.

Billing per the pricing page: pay-as-you-go at the provider’s own rate, no Hugging Face markup. PRO users and Team/Enterprise seats get monthly included credits; free accounts get none and must buy credits first. Amounts change, so read the page, not a blog post. Per-model and per-provider usage is on your Inference Providers settings.

If it fails

• 401 {"error":"Invalid username or password."} (AuthenticationError in the OpenAI SDK, HfHubHTTPError in huggingface_hub): the token is missing or invalid. Create a new fine-grained token with the Inference Providers permission and re-export it.
• 402 Payment Required: no credits left, or a free account with none bought. Add credits on the billing page.
• 503 Service Unavailable with a pinned provider: that provider is down. Drop the :provider suffix so the router can fail over, or pin another one listed on the model page.

---

Next: run a small model locally instead · pick an AI model · serve your own OpenAI-compatible endpoint with vLLM.