Engineering
Try a cheaper model before switching
Delegate a model change to Merlin in Linear. It runs the feature's eval cases on cheaper models through OpenRouter and opens a pull request with the comparison, or says none held up.
@Merlin from now on, when an issue asks you to try a cheaper model for a feature, run that feature's eval cases from acme/app through OpenRouter on the current model and on two or three cheaper candidates that support what the feature needs. Compare how many cases each gets right, what the run cost and the median latency. If a candidate gets within two cases of the current model, open a pull request switching to the cheapest one, with the comparison in its description. If none does, say so on the issue with the numbers and change nothing.
- Starts whenA teammate delegates an issue asking for a cheaper model for one feature
Merlin worksClaude Code or Codex runs the feature's eval cases from the repository on the current model and cheaper candidates through OpenRouter, then compares quality, cost and latency
- You getA pull request switching the model with the comparison in its description, linked on the issue
Tools
- Linear
- OpenRouter
- GitHub
Capabilities
- Integrations
- Your agent's MCP
How it works
From a model issue to a pull request with the numbers.
Delegate an issue that names the feature to make cheaper. It can name candidate models or leave the choice to Merlin.
Claude Code or Codex runs the feature's eval cases from
acme/appthrough OpenRouter on the current model and on cheaper candidates, then reads each generation's cost and latency. Progress shows in the issue's agent session.If a candidate holds up, a pull request switching to it is linked on the issue with the comparison in its description. If none does, Merlin posts the numbers on the issue and changes nothing.
Delegate an issue that names the feature to make cheaper. It can name candidate models or leave the choice to Merlin.
Claude Code or Codex runs the feature's eval cases from acme/app through OpenRouter on the current model and on cheaper candidates, then reads each generation's cost and latency. Progress shows in the issue's agent session.
If a candidate holds up, a pull request switching to it is linked on the issue with the comparison in its description. If none does, Merlin posts the numbers on the issue and changes nothing.
ENENG › Issues › ENG-482
Try a cheaper model for ticket triage
Triage sends every new support ticket to claude-sonnet-5. Try cheaper models on the cases in evals/triage and switch if they hold up.
Activity
Omardelegated the issue to Merlin·now
Properties
Todo
Omar
Merlin
Labels
Improvement
Before and after
Model switches get decided on your own cases, not on a hunch.
Without Merlin
An engineer reruns the evals on each candidate by hand, adds up the costs in a sheet and then makes the switch.
- 01Pick up the model issue
- 02Shortlist cheaper models
- 03Find the triage eval cases
- 04Run them on the current model
- 05Rerun them on each candidate
- 06Look up what each run cost
- 07Compare right answers, cost and latency
- 08Change the model id
- 09Run the tests
- 10Open a pull request with the numbers
- 11Link it on the issue
With Merlin
Delegate the issue and review a switch that comes with its own comparison.
- Model issue delegated to
MerlinNames the feature to make cheaper Runs the evals on Omar's MacBookThree models through OpenRouter
- Pull request with the comparisonLinked on the issue
Setup
Connect Linear and GitHub, then add OpenRouter to your agent.
Set these up once and anyone can ask for a cheaper model from an issue. Merlin never merges, so a person decides on the switch.
- 01Connect a machineRuntimeInstall the Merlin desktop app or CLI on a computer running Claude Code or Codex. You can also use a machine a teammate shares with your workspace.
- 02Connect LinearIntegrationsConnect your Linear workspace so issues can be delegated to Merlin.
- 03Add OpenRouter to your agentYour agent's MCPConnect the OpenRouter MCP server to Claude Code or Codex on the machine that runs this. Its credentials stay on that machine.
- 04Install the GitHub appIntegrationsPick the repositories Merlin can access, so it can read and link the pull requests there. Pushes use the machine owner's own Git access.
- 05Keep your code on the machineRuntimeMerlin reads the code in a local project on the runtime, so keep the repository checked out there.
- 06Keep eval cases next to the featureRuntime
Merlin scores each model against the cases your team keeps in the repository, such as
evals/triage/cases.jsonlwith the expected answer for each. Without them, it says so on the issue instead of guessing. - 07Size the key for the evalYour agent's MCP
Each case is a billable
send-messagecall. The key OpenRouter mints when you connect starts with a $10 spend cap and expires after 7 days (how it works), so raise the cap on the approval screen for a large eval set and reconnect once the key has expired. - 08Paste the setup promptGuidanceAn owner or admin sends the prompt below to @Merlin once, and it becomes workspace guidance for every future run.
Prompt
Tell Merlin how to try a cheaper model.
@Merlin from now on, when an issue asks you to try a cheaper model for a feature, run that feature's eval cases from acme/app through OpenRouter on the current model and on two or three cheaper candidates that support what the feature needs. Compare how many cases each gets right, what the run cost and the median latency. If a candidate gets within two cases of the current model, open a pull request switching to the cheapest one, with the comparison in its description. If none does, say so on the issue with the numbers and change nothing.
Once, when you paste it
- PromptYou paste the prompt to @Merlin
- GuidanceMerlin saves your rules as workspace guidance
Every time an issue is delegated to Merlin
- IntegrationLinear hands Merlin the issue
- RuntimeMerlin picks an available machine
- ContextThe run gathers context
- WorkClaude Code or Codex runs the evals on each model
- SteeringYour team steers it from the issue
- ResultMerlin links the pull request with the comparison
Related use cases
More engineering tasks you can hand to Merlin.
- AutomationsIntegrations
Engineering
Check the models in your code every month
Every month, Merlin checks each model id in your code against OpenRouter. You get expiring models and price changes in Slack, and a pull request that moves off anything expiring.
Day 1 of every month at
10:00A pull request for expiring models, in#eng - IntegrationsYour agent's MCP
Engineering
Find out why an LLM call failed
Mention Merlin in Slack with an OpenRouter generation id from your logs. You get why the call failed and the fix it would make, from the generation, its providers and your code.
@Merlinin#engwith a generation idThe cause and a fix, in the thread - Integrations
Engineering
Upgrade a dependency and fix what breaks
Delegate a dependency upgrade to Merlin in Linear. It bumps the version, fixes the breaking changes and opens one pull request linked on the issue.
Upgrade issue delegated to MerlinOne pull request, linked on the issue
Bring your coding agents into the tools your team already uses.
Let your whole team hand work to Claude Code and Codex today. Connect one machine and add Merlin to Slack, Linear or GitHub. 14-day free trial, no API keys, no card.