BYOE Guide

BYOE (Bring Your Own Endpoint) lets you register any OpenAI-compatible endpoint and benchmark it head-to-head against any model in the LLM Prover registry. Fine-tuned models, self-hosted deployments, Azure OpenAI custom URLs, GPU cloud deployments – if it speaks the OpenAI API format, you can register it. Once registered, your endpoint appears in the model picker on every comparison, evaluation, and benchmark run, alongside GPT, Claude, Grok, and the rest. The result is reproducible, exportable evidence that your model performs the way you claim – not anecdote.

What you can register

Any endpoint that accepts requests in the OpenAI chat completions format can be registered. The endpoint does not need to be hosted by OpenAI. It needs to accept the same request structure and return the same response structure. Most modern inference servers and cloud providers support this format.

• Fine-tuned models. OpenAI fine-tunes (ft:gpt-4o-mini:org:name:id), or any provider that exposes fine-tunes via an OpenAI-compatible URL.
• Self-hosted models. Ollama, vLLM, LM Studio, llama.cpp with an OpenAI-compatible server.
• GPU cloud deployments. Replicate, Modal, RunPod, Together AI, Groq, Fireworks, and similar.
• Azure OpenAI. Custom deployment URLs with API key auth.
• Any other OpenAI-compatible inference endpoint.
• Chained workflows and pipelines. Any system that accepts a prompt and returns a response – a RAG pipeline, an agentic workflow, a multi-step backend – can be registered as a BYOE endpoint. You own the workflow. LLM Prover owns the measurement. The pipeline runs as a black box: prompt in, output out, scored by the same rubrics as any other model.

Benchmarking a pipeline end to end

BYOE is not limited to individual models. Any system that accepts a prompt and returns a response can be registered as an endpoint – including a full chained workflow, a RAG pipeline, or an agentic backend. From LLM Prover’s perspective it is just another provider. From your perspective you are benchmarking your entire pipeline as a black box.

This matters because most pipeline evaluation approaches require you to instrument every layer: log the retrieval step, log the reranking step, log the generation step. BYOE skips all of that. You fire the same input conditions at your pipeline endpoint that a real user would send, and you measure what comes out. The rubric defines what a good output looks like. The benchmark runs it repeatedly and tracks whether that standard holds.

Two things this enables that unit testing individual models does not:

• E2E quality signal. A model that scores 95 in isolation may score 72 when it is the generation layer of a RAG pipeline with a poorly structured knowledge base. The E2E benchmark catches that. The unit test does not.
• Pipeline vs model comparison. Register your pipeline alongside individual frontier models in the same benchmark run. If your pipeline – which includes retrieval, reranking, and a custom system prompt – outperforms a raw frontier model on your task, you have evidence that the pipeline architecture is adding value. If it does not, you know where to look.

The only requirement is that your pipeline exposes an OpenAI-compatible endpoint. If it accepts a prompt in the standard chat completions format and returns a response, registration takes under a minute.

Registering an endpoint

Endpoints are registered from the Endpoints section in the Config sidebar. Each endpoint needs a name, a base URL, a model ID, and credentials. The name is what appears in the model picker. The base URL is the root of your OpenAI-compatible API. The model ID is the identifier your endpoint expects in the model field of the request.

1. Navigate to Endpoints in the sidebar (Config section)
2. Click “+ Add Endpoint”
3. Fill the form:
   • Name. What appears in the model picker and run results (e.g. “Llama 3.1 fine-tune v2”)
   • Base URL. Root URL of your endpoint (e.g. https://api.groq.com/openai/v1)
   • Model ID. The model identifier your endpoint expects (e.g. llama-3.1-8b-instant or ft:gpt-4o-mini:org:name:id)
   • Auth type. Bearer token, API key, or None
   • Credential. Your API key or token – stored encrypted, never logged
   • Pricing (optional). Input and output cost per 1k tokens. If your endpoint returns token usage, LLM Prover calculates cost automatically. If it does not return usage, enter rates here or cost columns will show N/A.
4. Check the BYOE terms checkbox – you are responsible for API costs on your provider account
5. Click “Test connection” to fire a test call before saving – shows latency and a sample response
6. Click “Register Endpoint” to save

The endpoints table

Every registered endpoint appears as a row in the Endpoints table. The table shows the endpoint name, base URL, model ID, status, registration date, and last tested date. Status reflects the most recent connection test. Click any row to open the endpoint drawer and edit or re-test it.

• Active (green dot) – last connection test passed
• Error (red dot) – last connection test failed
• Unverified (grey dot) – never tested

Bulk delete is available via checkbox selection.

Using your endpoint in runs

Once registered and active, your endpoint appears in the model picker on every comparison, evaluation, and benchmark form – alongside GPT, Claude, Grok, and every other model in the registry. It behaves identically to any other model. You select it the same way, it runs in the same pass, and its results appear in the same table. The only difference is that the API call goes to your endpoint instead of a managed provider.

• Appears in the model picker grouped under “Your Endpoints”
• Available in Comparisons, Evaluations, and Benchmarks
• Runs in parallel with other selected models – same prompt, same parameters
• Results include cost (if pricing is configured or the endpoint returns usage), latency, and quality score
• Diagnostics runs on your endpoint the same as any other model
• “New from this” carries your endpoint into subsequent runs

Pricing and cost tracking

Cost tracking for BYOE endpoints works the same as for managed models, with one condition: LLM Prover needs token usage data to calculate cost. Most endpoints return token usage in the response. If yours does, and you have entered per-token rates in the Pricing section of the endpoint form, cost is calculated automatically and appears in the cost column alongside managed model costs. If your endpoint does not return usage data, cost columns show N/A for that endpoint.

• Enter input and output cost per 1k tokens in the Pricing section of the endpoint form (optional, collapsible)
• If the endpoint returns token usage: cost = (input tokens / 1000 * input rate) + (output tokens / 1000 * output rate)
• If the endpoint does not return usage: cost column shows N/A; latency and quality still work normally
• Cost data feeds into the cost estimator when the endpoint is selected in a benchmark form
• Update pricing at any time by clicking the endpoint row and editing the form

Editing and re-testing

Click any endpoint row to open the edit drawer. You can update the name, base URL, model ID, credential, and pricing at any time. Changes take effect on the next run – existing run history is not affected.

• All fields are editable: name, base URL, model ID, pricing
• Credential field. Leave blank to keep the existing credential. Enter a new value to replace it.
• “Test connection” in edit mode tests the current saved endpoint configuration, not unsaved form values
• Status updates to Active or Error after each test
• Deleting an endpoint removes it from the model picker immediately. Existing benchmark runs that used it retain their historical data, but the endpoint will not appear in future runs.

Tier limits

BYOE is available on Pro and above. The number of endpoints you can register depends on your tier. See the models page for the current cap on your tier – caps may change as the product evolves.

What’s next