Bring Your Own Model Setup Guide

Use your own API keys to add models alongside TruVerifAI's four default providers. Each guide below covers where to get the key, what model IDs work, and what to paste into the BYOM form.

Open Bring Your Own Model →

Provider comparison

9 providers · 9 wire protocols
ProviderSetupModels supported
OpenAI directEasyAll chat models
Anthropic directEasyAll chat models
Google AI StudioEasyGemini family
Together AIEasyLlama, Mistral, Qwen, ...
OpenRouterEasy200+ from many providers
FireworksEasyOpen-weight chat models
Ollama (local)EasyWhatever you pull locally
AWS BedrockMedium · IAMNova, Llama, Mistral, Titan, Claude, custom imports
GCP Vertex AIHard · GCPGemini, Claude, DeepSeek, Llama, Mistral

Which wire protocol?

The wire protocol tells TruVerifAI how to talk to your model's host. Pick it by where the model runs:

  • A first-party account (OpenAI, Anthropic, or Google AI Studio): pick the matching named protocol.
  • An OpenAI-compatible aggregator (Together, OpenRouter, Fireworks, Groq, a local Ollama): pick OpenAI-compatible and set the Base URL the host documents. For example, https://api.together.xyz/v1 for Together AI.
  • AWS Bedrock: use AWS Bedrock for any model (Nova, Llama, Mistral, Titan, Cohere, custom imports). Use AWS Bedrock — Claudeonly for Claude models; it routes through Claude's native API for the full feature set.
  • GCP Vertex AI: use GCP Vertex AI — Gemini only for Gemini, GCP Vertex AI — Claude for Claude on Vertex, and GCP Vertex AI — Other models for anything else in Model Garden (DeepSeek, Llama, Mistral, ...).

OpenAI direct

  1. 1Sign in to platform.openai.com.
  2. 2API keys → Create new secret key.
  3. 3Copy the key (starts with sk-...).
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: OpenAI (api.openai.com).
  6. 6API Key: paste your sk-... key.
  7. 7Model ID: e.g. gpt-4o, gpt-4-1106-preview, or any chat model_id from platform.openai.com/docs/models.
  8. 8Click Test & add.

Anthropic direct

  1. 1Sign in to console.anthropic.com.
  2. 2API Keys → Create Key.
  3. 3Copy the key (starts with sk-ant-...).
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: Anthropic (api.anthropic.com).
  6. 6API Key: paste your sk-ant-... key.
  7. 7Model ID: e.g. claude-3-5-sonnet-20241022, claude-3-opus-20240229, or any chat model_id from docs.anthropic.com.
  8. 8Click Test & add.

Google AI Studio

  1. 1Go to aistudio.google.com → Get API key.
  2. 2Create API key in new project (or pick existing).
  3. 3Copy the key.
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: Google AI Studio.
  6. 6API Key: paste your key.
  7. 7Model ID: e.g. gemini-2.0-flash, gemini-1.5-pro, or any Gemini chat model_id from ai.google.dev/models.
  8. 8Click Test & add.
Note: Google AI Studio is the simplest, direct Gemini API (it has a free tier, with paid usage beyond it). Vertex AI is the enterprise option. See the Vertex section below.

Together AI

  1. 1Sign in to api.together.xyz.
  2. 2Settings → API Keys → Generate key.
  3. 3Copy the key.
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: OpenAI-compatible (Together, OpenRouter, ...).
  6. 6API Key: paste your Together key.
  7. 7Base URL: https://api.together.xyz/v1
  8. 8Model ID: use Together's catalog format, e.g. meta-llama/Llama-3.1-70B-Instruct-Turbo. Find full IDs at docs.together.ai.
  9. 9Click Test & add.

OpenRouter

  1. 1Sign in to openrouter.ai.
  2. 2Keys → Create Key.
  3. 3Copy the key (starts with sk-or-...).
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: OpenAI-compatible.
  6. 6API Key: paste your sk-or-... key.
  7. 7Base URL: https://openrouter.ai/api/v1
  8. 8Model ID: use OpenRouter's routing prefix format, e.g. anthropic/claude-3-5-sonnet or meta-llama/llama-3.1-70b-instruct. Full catalog at openrouter.ai/models.
  9. 9Click Test & add.
Note: OpenRouter is a unified gateway to 200+ models from many providers. Useful for trying obscure or open-weight models without managing each provider's account.

Fireworks

  1. 1Sign in to fireworks.ai.
  2. 2Account → API Keys → New API key.
  3. 3Copy the key.
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: OpenAI-compatible.
  6. 6API Key: paste your Fireworks key.
  7. 7Base URL: https://api.fireworks.ai/inference/v1
  8. 8Model ID: use Fireworks' account-scoped format, e.g. accounts/fireworks/models/llama-v3p1-70b-instruct. Catalog at fireworks.ai/models.
  9. 9Click Test & add.

Ollama (local)

  1. 1Install Ollama from ollama.com.
  2. 2Run a model locally: ollama pull llama3.1 && ollama run llama3.1
  3. 3Confirm the local OpenAI-compatible endpoint is reachable at http://localhost:11434/v1
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: OpenAI-compatible.
  6. 6API Key: any non-empty string (Ollama ignores it but the form requires a value).
  7. 7Base URL: http://localhost:11434/v1
  8. 8Model ID: the local model name (e.g. llama3.1, mistral, qwen2.5-coder).
  9. 9Click Test & add.
Note: Local Ollama only works when TruVerifAI is running on the same machine as Ollama. For shared usage, expose Ollama behind a tunnel (cloudflared, ngrok) and use the tunnel URL as the Base URL.

AWS Bedrock

The picker has two AWS options: AWS Bedrock (the Converse API, which works for any Bedrock model: Llama, Nova, Mistral, Titan, Cohere, custom imports) and AWS Bedrock — Claude (Claude through the native Anthropic API, for full Claude features). Both use the same AWS credentials.

  1. 1In the AWS console: Bedrock → Model access → enable the models you want. Anthropic Claude needs a short use-case form; Amazon Nova and Meta Llama are usually instant.
  2. 2IAM → create a user with an inline policy allowing bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, bedrock:Converse, bedrock:ConverseStream.
  3. 3Create an access key for that user (use case: application running outside AWS).
  4. 4In TruVerifAI: Bring Your Own Model → Add custom model.
  5. 5Wire protocol: AWS Bedrock, or AWS Bedrock — Claude for a Claude model.
  6. 6Paste the Access Key ID and Secret Access Key. Leave Session Token blank unless you use temporary STS credentials.
  7. 7Region: the region where you enabled the model, e.g. us-east-1.
  8. 8Model ID: the Bedrock model id, e.g. amazon.nova-lite-v1:0. Newer models need a cross-region inference profile. Prefix the id with the region group, e.g. us.anthropic.claude-sonnet-4-5-20250929-v1:0.
  9. 9Click Test and add.
Note: A failure mentioning on-demand throughput means the model needs a cross-region inference profile. Prefix the model id with us., eu. or apac. for your region group.

GCP Vertex AI

The picker has three Vertex options: GCP Vertex AI — Gemini only (Gemini models), GCP Vertex AI — Claude (Claude on Vertex), and GCP Vertex AI — Other models (DeepSeek, Llama, Mistral and other OpenAI-compatible Model Garden models). All three authenticate with the same service-account JSON key.

  1. 1In the GCP console: enable the Vertex AI API for your project.
  2. 2For Claude or an Other-models pick: Vertex AI → Model Garden → open the model card → Enable (one-time). Gemini base models need no enablement.
  3. 3IAM and Admin → Service Accounts → create one with the Vertex AI User role (roles/aiplatform.user).
  4. 4Create a JSON key for that service account and download it.
  5. 5In TruVerifAI: Bring Your Own Model → Add custom model.
  6. 6Wire protocol: GCP Vertex AI — Gemini only, GCP Vertex AI — Claude, or GCP Vertex AI — Other models.
  7. 7Paste the entire service-account JSON, plus your project ID.
  8. 8Region: for Gemini use a regional one like us-central1, or global for newer 3.x models. For Claude use the region its Model Garden card lists, usually us-east5.
  9. 9Model ID: the bare model name. gemini-2.5-flash for Gemini, or claude-haiku-4-5@20251001 for Claude (note the @date suffix). For an OpenAI-compatible Model Garden model, use the publisher/model-id form: keep the publisher, but drop the publishers/ and /models/ segments the GCP console adds. For example, the console shows publishers/meta/models/llama-4-maverick-17b-128e-instruct-maas, so you enter meta/llama-4-maverick-17b-128e-instruct-maas. The bare model name without the meta/ publisher does NOT work.
  10. 10Click Test and add.
Note: Claude on Vertex needs a non-zero token quota for the model in your GCP project. A fresh project often starts at 0. Request an increase under IAM and Admin → Quotas before adding the BYOM, or Test and add will fail with a rate-limit error.

Use a model as the synthesizer

The synthesizer is the model that merges the panel's responses into the single final answer. By default that's the platform model, but you can point it at one of your own models. The four-model panel still runs; your model just writes the final synthesis, on your own key at zero variable credits.

  1. 1Add and test the model first (any guide above), so it's active in Bring Your Own Model.
  2. 2Open the Models page → the Synthesizer model card → Change model.
  3. 3In the chooser, scroll to Your modelsat the bottom and pick one. Eligible models are selectable; ones that can't synthesize yet are shown greyed with the reason.
  4. 4Save your preferences. Your model now writes the final answer in chat, over the API, and for your agent (an individual API call can still override it per request).

Test MCP compatibility (recommended)

Next to each of your models (in the chooser and on the Bring Your Own Model page) is a Test MCP compatibility button. It runs a few quick calls (~3) on your own key to confirm the model reliably produces the structured final answer the MCP path needs, then shows a badge:

  • Validated for MCP: passed; trusted to synthesize everywhere, including MCP.
  • Not tested: never run. Chat and API synthesis work; MCP synthesis uses the default synthesizer until you validate.
  • Not MCP-compatible: the model didn't produce the structured output cleanly. Chat and API still work; MCP uses the default.
Note: A model can be picked as the synthesizer only while it's active and supports streaming. The compatibility test runs on your credentials (a few calls at your provider's rate); the synthesis itself never costs TruVerifAI credits.

Common pitfalls

  • Bedrock, on-demand throughput error: newer models reject the bare model id. Prefix it with the region group (us., eu., apac.).
  • Bedrock, model not enabled: the model must be enabled for that exact region under Bedrock → Model access.
  • Vertex, use the bare model name: gemini-2.5-flash, not the publishers/google/models/... path the console shows.
  • Vertex, region matters: newer Gemini models are often only on global; Claude on Vertex is region-locked, usually us-east5. A not-servable-in-region error means switch to the region the Model Garden card lists.
  • Vertex, Claude quota: a new GCP project often has a zero token quota for Claude. Request an increase before testing.
  • Vertex Other Models, id format: use the publisher/model-id form, e.g. meta/llama-4-maverick-17b-128e-instruct-maas. Keep the publisher, but drop the publishers/ and /models/ segments: the console's publishers/meta/models/llama-4-maverick-17b-128e-instruct-maas becomes meta/llama-4-maverick-17b-128e-instruct-maas. The bare model name without the publisher does NOT work.
  • Vertex Other Models, region: global is correct for DeepSeek V3.2; us-west2is common for Llama / Mistral MaaS. Check the Model Garden card's code snippet for the region the model actually serves on.