Test AI models on fiction yourself, then build with the winner
Last checked against the product November 10, 2026
A leaderboard tells you which model scored best on average. It cannot tell you which one you will like reading. A bake-off lets you find out. Pick two to five models, from Novelmint’s own, from the wider OpenRouter list, or from your own API key, and each one writes the same scenes from our public fiction benchmark. A panel of judges scores every passage blind, and you read them side by side, names hidden if you want. When one stands out, a single click makes it the model your series builds with.
Key takeaways
- A bake-off runs Novelmint’s fiction benchmark on two to five models you choose: Novelmint’s, any from the OpenRouter list, or models on your own API key.
- Every model gets the same brief for each scene, drawn from the benchmark’s standard scenes, so the comparison is like for like.
- The benchmark’s own judge panel scores each passage blind, and a judge never scores a model from its own maker.
- You read the passages side by side, can hide the model names until you pick, and can save the whole run as a file.
- One click on "Use for Builds" makes the winner the model for that series, or your Studio default for every series.
What it does
What a bake-off gives you
Up to five models at once
Mix Novelmint’s measured models, models from the "More models" list that have no score yet, and base models your own key can reach.
The same scenes for everyone
Each round, every model writes a few of the benchmark’s standard scenes, the ones with the fewest scores so far. Choose how many rounds to run at once.
Blind scoring
The same panel of judges as the public benchmark scores each passage without knowing which model wrote it, and never one from its own maker. A score runs 0 to 100: how well the scene does what it tests, plus how closely it kept to the brief.
Blind reading, for you too
Tick "Hide model names until I pick" and the passages appear as Model A, Model B and so on, so you judge the writing before the reputation.
Keep the evidence
Save a run as a file, with a key to which model is which. Your past bake-offs stay listed on the tab.
Straight into your builds
Press "Use for Builds" on the passage you liked. From a series it sets that series’ model; from your Studio it sets the default for every series.
How to use it
Running one
Open the Bake-off tab
It sits on the AI Models page, both in your Studio and on every series.
Pick two to five models
Search Novelmint’s models, the wider OpenRouter list, and the models on any API key you have saved under Account, then AI keys.
Choose how many rounds and check the price
The tab shows the price for the run before it starts, and how many credits you have.
Run it and read
When the run finishes, read each scene’s passages side by side, with word counts and scores. Hide the names if you want to read them blind.
Use the winner
Press "Use for Builds" on the one you want. Your next chapter build uses it.
Example
What a result looks like
You ask
One benchmark scene: two estranged sisters meet at their father’s funeral, and neither says what she came to say.
You get back
- Model A: 231 words, score 78. Lets the silence carry the scene and keeps strictly to the brief.
- Model B: 214 words, score 71. Strong dialogue, but one sister says the unsaid thing aloud in the last line.
- Model C: 226 words, score 64. Vivid setting, and the emotion is told more than shown.
- You pick Model A, the names are revealed, and "Use for Builds" makes it your series’ model.
An illustration with an invented scene, invented scores and blind labels, to show the shape of a result. Your run uses the benchmark’s real scenes and the judges’ real scores.
Questions
Frequently asked
- How can I compare AI models for writing fiction?
- In Novelmint, open the Bake-off tab on the AI Models page and pick two to five models. Each writes the same standard scenes from the public fiction benchmark, a panel of judges scores every passage blind, and you read them side by side.
- Does a bake-off use my own book?
- No. Every model writes the benchmark’s standard scenes, so the results compare like with like and your manuscript is not part of the run. To see how a model handles your own chapters, pick it for a series and build.Choose your AI model
- Can I test a model on my own API key?
- Yes. Any base model your saved key can reach can join a bake-off, and its writing runs on your key. Models you have fine-tuned yourself are left out, so the judges always see a model as its maker built it.Bring your own AI key
- Who judges the passages?
- The same panel of judge models as Novelmint’s public benchmark. They score each passage without seeing which model wrote it, and a judge never scores a model from its own maker.
- What does a bake-off cost?
- The judging, and any writing by Novelmint’s models, cost credits at what the AI costs Novelmint, with no markup. Writing on your own key costs no credits. You see the price before you start, and you are never charged more than it.Pricing
- Do my results become public?
- Your scores, never the passages, can help rank models on the public benchmark. You can turn that off on the tab, and turning it off also takes back the scores from your earlier runs.
- How do I use the model that won?
- Press "Use for Builds" on its passage. From a series, that series now builds with it. From your Studio, it becomes the default for every series that has no models of its own.
What this page does not claim
- A bake-off runs on the benchmark’s standard scenes, not on your manuscript.
- A few scenes are a sample. A model that wins a short run is not guaranteed to suit every chapter of your book.
- Judges score craft and fidelity to the brief. Whether you like the voice is still your call, which is why you read the passages yourself.
- The scenes, scores and labels in the example on this page are invented. They are not output from a real run.
Keep reading
The benchmark behind it
A bake-off is the public fiction benchmark, run by you on the models you care about. The scores your runs produce can help rank models for everyone, which is why judging is priced at what it costs.
Related
Plan a book with Minty
Talk the story through and watch it become chapters and scenes on your Timeline.
Chapter builds
Turn a chapter’s beats into prose, written from your plan, your bible, and your voice.
Bring your own AI key
Build chapters on your own Anthropic, OpenAI, Google, xAI or OpenRouter key, or on the AI your credits include. Your choice, per series.
Choose your AI model
Pick the AI model that writes your series, compared on measured writing quality and cost, and pin a different one to any single beat.
Read the evidence, then choose your writer.
No card. Your first chapter is free.