How do you use the Hugging Face Inference API and the InferenceClient for production deployments?

Running large models locally requires substantial GPU infrastructure. The Hugging Face Inference API offers serverless inference for thousands of public models — you send HTTP requests and receive predictions without managing any compute. The huggingface_hub library's InferenceClient provides a typed Python interface over this API, including an OpenAI-compatible messages format for chat models.

# pip install huggingface_hub
from huggingface_hub import InferenceClient

# Uses HF_TOKEN environment variable
client = InferenceClient('mistralai/Mistral-7B-Instruct-v0.3')

# ── Text generation
response = client.text_generation(
    'Explain LLMs in one sentence.',
    max_new_tokens=100,
    temperature=0.5,
)
print(response)

# ── Chat completion (OpenAI-compatible interface)
chat_response = client.chat_completion(
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user',   'content': 'What is RAG?'},
    ],
    max_tokens=200,
    temperature=0.3,
)
print(chat_response.choices[0].message.content)

# ── Streaming
for token in client.text_generation('Write a poem about AI:', stream=True,
                                      max_new_tokens=150):
    print(token, end='', flush=True)

# ── Embedding
embed_client = InferenceClient('BAAI/bge-small-en-v1.5')
vector = embed_client.feature_extraction('Hello world')
print(len(vector))  # embedding dimension

# ── Image classification
img_client = InferenceClient('google/vit-base-patch16-224')
labels = img_client.image_classification(
    'https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/1600px-Cute_dog.jpg'
)
print(labels[:3])  # top 3 predicted labels with scores