Hercules AI Gateway

Use the Hercules AI Gateway to build AI features in your app like chatbots, content generation, image generation, speech, transcription, and summarization. Hercules AI Gateway is built into all apps and works out of the box. You can use all the most popular models from OpenAI, Anthropic, and Google without needing to set up API keys.

AI Gateway customer support chatbot widget example

How do I set it up?

Ask Hercules to build you an app that uses AI! Hercules will use Hercules AI Gateway by default e.g.

Setup Hercules AI Gateway

Build a customer support chatbot for my app. Have it expand from a widget in the bottom right

What can I build with AI Gateway?

Chatbots: Customer support chatbot, AI coach
Content generation: Write descriptions, emails, summaries
Image generation: Create images from a text prompt
Voice and audio: Turn text into speech, transcribe audio and video to text
Analysis: Extract insights from text, categorize data
Summarization: Condense long documents or articles
Personalization: Generate custom recommendations

Can I generate images and audio?

Yes. The AI Gateway handles images and audio through the same setup, with no extra API keys.

Image generation: Create images from a text prompt
Text to speech: Turn written text into spoken audio, with a choice of voices
Speech to text: Transcribe audio and video files into text

Ask Hercules to add these to your app:

Add image generation

Add a feature that turns a text prompt into an image

Add text to speech

Add a button that reads my articles aloud

Add transcription

Let users upload an audio file and get back a text transcript

Can my app make decisions, like sorting or flagging content?

Yes. Use AI Decisions with Jev to classify, route, moderate, and score content. It is faster and cheaper than a chatbot model for these tasks.

What models can I choose?

Any current model from Anthropic, Cloudflare Workers AI, Google AI Studio, or OpenAI. The gateway passes the model string straight through to the provider, so new releases from these providers work as soon as the provider ships them. There is no allowlist to wait on. The table below is a starting point, not a complete list. Extra large and large models are best for complex work. Small models are faster and cost less. Tell Hercules the model you want by name, or use the model string when working with the API directly (see the technical FAQ below).

Provider	Model	Size	Model string	Best for
OpenAI	GPT-6 Astra	Extra large	`openai/gpt-6-astra`	The most advanced OpenAI model for reasoning and agent work
OpenAI	GPT-6 Sol	Large	`openai/gpt-6-sol`	Complex reasoning, coding, and analysis
OpenAI	GPT-6 Luna	Small	`openai/gpt-6-luna`	Everyday tasks at low cost (the default)
Anthropic	Claude Fable 5.1	Extra large	`anthropic/claude-fable-5-1`	The most demanding reasoning and agent work
Anthropic	Claude Opus 5.5	Large	`anthropic/claude-opus-5-5`	Deep reasoning and long, multi-step tasks
Anthropic	Claude Sonnet 5.5	Medium	`anthropic/claude-sonnet-5-5`	Strong coding and agent work at lower cost
Anthropic	Claude Haiku 4.5	Small	`anthropic/claude-haiku-4-5`	Fast, low-cost chat and simple tasks
Google	Gemini 3.1 Pro	Large	`google/gemini-3.1-pro-preview`	Advanced multimodal reasoning (preview)
Google	Gemini 3.8 Flash	Medium	`google/gemini-3.8-flash`	Fast agentic and coding tasks
Google	Gemini 3.5 Flash-Lite	Small	`google/gemini-3.5-flash-lite`	High-volume tasks that need instant answers
Workers AI	GLM 5.3 Flash	Medium	`workers-ai/@cf/zai-org/glm-5.3-flash`	Very long context and vision at a low price
Workers AI	Qwen 3.8 27B	Medium	`workers-ai/@cf/qwen/qwen3.8-27b`	Open source reasoning and tool use

What model does Hercules use by default?

By default, Hercules AI Gateway uses GPT-6 Luna by OpenAI. It’s fast, cheap, and very accurate.

How do I choose another model?

If you’d like to use another model, just tell Hercules the model and/or provider you want to use:

Use Claude Haiku by Anthropic
Use Gemini 3.8 Flash by Google

Can I use a model that is not in the table?

Yes. Use the provider’s own model id after the provider prefix, for example anthropic/claude-fable-5-1 or openai/gpt-6-astra. If the provider serves the model, the gateway serves it. If the provider rejects the id, the gateway returns the provider’s error. Workers AI model ids keep their own @cf/ namespace, so the full string has two parts after the prefix: workers-ai/@cf/qwen/qwen3.8-27b. Any model in the Cloudflare Workers AI catalog works this way. Models from providers other than Anthropic, Cloudflare Workers AI, Google AI Studio, and OpenAI are not available.

What input/output is supported?

	Input	Output
Today	Text, images, PDFs, audio, video	Text, images, voice/audio
Planned	Realtime voice	Video

How is billing handled?

AI Gateway usage is billed separately through Hercules Cloud. It’s a separate line item from the Hercules App Builder Agent (what you use to build your app). We bill the model provider’s rate plus a 7.5% platform fee. For instance, if you use $10 of OpenAI, you will be billed $10.75 of Hercules Cloud. Enterprise plans can have a custom platform fee. Contact sales@hercules.app for details.

Can I see what each AI request costs?

Yes, for most models, when your app uses the Responses or Messages format. Their text responses include what the request cost, in dollars, including the 7.5% platform fee. It is the same amount billed to your Hercules Cloud balance. Ask Hercules to show it in your app:

Show AI cost per answer

Show how much each AI answer cost, below the answer

Chat Completions responses (the default format) do not currently include the cost. For those, Hercules can estimate it from the token counts in the response at the model’s price plus the 7.5% platform fee. For exact amounts, go to Settings → Billing → Hercules Cloud.

Do I need my own API keys?

No. This is all managed by Hercules out of the box through the HERCULES_API_KEY. If you want to manage billing yourself, you need to ask Hercules to remove the Hercules AI Gateway, then configure the API key and model yourself.

Additional FAQ

What's the difference between AI Gateway and the Hercules App Builder Agent?

The Hercules App Builder Agent builds your app. AI Gateway lets your users use AI features inside your published app.They are also currently billed separately. Hercules App Builder Agent bills against your monthly AI credits. Hercules AI Gateway bills against your Hercules Cloud credits.

How does the Hercules AI Gateway work technically?

Use the OpenAI SDK with Hercules configuration:

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://ai-gateway.hercules.app/v1",
  apiKey: process.env.HERCULES_API_KEY,
});

const response = await openai.chat.completions.create({
  model: "openai/gpt-6-luna",
  messages: [{ role: "user", content: prompt }],
});

The HERCULES_API_KEY is automatically available in your app’s environment.

Which API formats does it support?

Chat Completions (OpenAI SDK, shown above): the default
Responses API (OpenAI SDK): openai.responses.create({ model: "openai/gpt-6-luna", input })
Messages API (Anthropic SDK), for anthropic/... models:

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({
  baseURL: "https://ai-gateway.hercules.app",
  apiKey: process.env.HERCULES_API_KEY,
});

For most models, Responses and Messages text responses include usage.cost: what the request cost in US dollars, including the 7.5% platform fee. When streaming, it is on the final chunk that carries usage.Chat Completions responses do not currently include usage.cost. Estimate their cost from the usage token counts at the model’s price plus the 7.5% platform fee, and treat your Hercules Cloud usage as the exact amount.

How do I generate images, speech, and transcripts?

Use the same client. Switch the method and model:

// Image generation
const image = await openai.images.generate({
  model: "openai/gpt-image-2.5-flare",
  prompt: "A serene mountain landscape at sunset",
});

// Text to speech
const speech = await openai.audio.speech.create({
  model: "openai/gpt-4o-mini-tts",
  voice: "coral",
  input: "Hello from Hercules",
});

// Speech to text
const transcript = await openai.audio.transcriptions.create({
  model: "openai/gpt-4o-transcribe",
  file: audioFile,
});

Is there rate limiting?

Yes. Hercules handles rate limiting automatically. If you need higher limits, please contact support.

Can I stream responses?

Yes. Hercules can set up your chatbot to stream responses in real-time as they come through

How do I monitor AI Gateway usage?

View usage in your billing dashboard under Hercules Cloud.

Is there a size limit on AI requests?

Yes. Each chat or text request can be up to 8 MB. Audio transcription and image edit uploads can be up to 50 MB.Files sent inside a request, like a PDF or an image, grow by about a third when they are encoded, so a single request can carry roughly 6 MB of file. Larger requests are rejected with “Request body is too large”.For a large PDF, split it into smaller batches of pages and send each batch as its own request. You can ask Hercules to set this up for you.

What's the difference between this and AI Image Generation?

AI Gateway image generation runs inside your published app, for your users, at runtime. The AI Image Generation feature is Hercules creating images for your app while you build it (logos, banners, illustrations).

Get Started

Hercules App Builder Agent

Build your app

Publish your app

Advanced

App management

Hercules AI Gateway

How do I set it up?

What can I build with AI Gateway?

Can I generate images and audio?

Can my app make decisions, like sorting or flagging content?

What models can I choose?

What model does Hercules use by default?

How do I choose another model?

Can I use a model that is not in the table?

What input/output is supported?

How is billing handled?

Can I see what each AI request costs?

Do I need my own API keys?

Additional FAQ

​How do I set it up?

​What can I build with AI Gateway?

​Can I generate images and audio?

​Can my app make decisions, like sorting or flagging content?

​What models can I choose?

​What model does Hercules use by default?

​How do I choose another model?

​Can I use a model that is not in the table?

​What input/output is supported?

​How is billing handled?

​Can I see what each AI request costs?

​Do I need my own API keys?

​Additional FAQ

How do I set it up?

What can I build with AI Gateway?

Can I generate images and audio?

Can my app make decisions, like sorting or flagging content?

What models can I choose?

What model does Hercules use by default?

How do I choose another model?

Can I use a model that is not in the table?

What input/output is supported?

How is billing handled?

Can I see what each AI request costs?

Do I need my own API keys?

Additional FAQ