SpecJudge: which AI model is right-sized for your project? Ask your specs

<p>When you finish planning a project and it's time to actually build it, there's a small decision that quietly costs money: <strong>which model do you use?</strong></p> <p>Pick one that's too powerful for the job and you pay for capability the project never touches. Pick one that's too weak and it can't deliver — so you pay and get nothing.</p> <p>I got tired of guessing, so I built a tool that answers it. It's called SpecJudge, it's open source, and it runs entirely on your machine.</p> <h2> The idea </h2> <p>If you work with <strong>Spec-Driven Development</strong> (SDD), you define your project first — constitution, spec, tasks — and only then start implementing. That moment, right before you write the first AI-assisted line, is when you have the richest possible description of the work and have spent exactly nothing on it.</p> <p>SpecJudge uses that description to answer the model question at the cheapest possible moment.</p> <p>It reads your SDD artifacts, and a <strong>local model running through Ollama</strong> estimates how demanding the project actually is. That demand is crossed against a catalog of models, and you get a ranked podium of what fits best — each with its price.</p> <h2> The part I care about most: right-sizing </h2> <p>SpecJudge does not recommend the cheapest model. It doesn't recommend the most powerful one either. It recommends the one that's right-sized for your project.</p> <p>The podium ranks by fit. Price is shown for every model, but it only breaks ties between models that fit equally well. A cheaper model never wins over one that does the job better — because recommending something that can't do the job is the most expensive mistake of all.</p> <p>Every model gets a rating on a fixed scale:</p> <ol> <li>good Capable enough — the sweet spot</li> <li>overkill Capable, but you're paying for headroom you won't use</li> <li>fair Falls somewhat short</li> <li>poor Not capable enough</li> </ol> <h2> Local by design </h2> <p>The judge runs on your machine. Your specs — your business logic, your design decisions — never touch a third-party service, and figuring out which model to buy costs you nothing in API calls. The optional browser report (--open) is a self-contained HTML file that loads nothing from the network.</p> <p>This isn't a nice-to-have bolted on afterwards; it's the first principle of the project.</p> <h2> Quick start </h2> <p>You'll need Python 3.11+ and Ollama with at least one local model.<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>ollama pull llama3.1:8b <span class="c"># a judge to evaluate your project</span> pip <span class="nb">install </span>specjudge specjudge /path/to/your/project </code></pre> </div> <p>On first run it lists your local models, asks which one to use as the judge, and remembers your choice. Add --open for a visual matrix in your browser, filterable by model family and by open source.</p> <p><code>specjudge . --open</code></p> <h2> It's honest about what it doesn't know </h2> <p>A spending recommendation built on thin data is worse than none. So SpecJudge is explicit about how much you can trust it:</p> <ul> <li>Sufficient — constitution, spec and tasks present: reliable recommendation.</li> <li>Scarce — artifacts missing or thin: recommendation issued, with a warning.</li> <li>Insufficient — nothing to evaluate: no recommendation at all.</li> </ul> <p>And if no model in the catalog is capable enough for your project, it says so instead of recommending one anyway. Data states and exit codes are distinguishable, so you can wire it into CI.</p> <h2> Open source, and built to be maintained by the community </h2> <p>Models and prices change every few weeks. That's the whole reason this project needs contributors.</p> <p>The most valuable contribution requires zero Python — the catalog lives in plain YAML, deliberately separate from the code. Add