What Benchmate does not check (yet)

The last post listed what Benchmate, the AI assistant in Ohmeris, checks before it gives you a circuit. This one covers the other half: what it does not check, and why its summary says so plainly instead of sounding confident.

Most AI drafts end with a line like "Circuit function was not simulated or verified." That line is usually true, and you should read it.

1. Whether the circuit does what you asked

The structural checks confirm that a draft is a well-formed circuit. Pins exist, wires land on them, nothing is shorted to ground, values parse, and the drawing is readable. None of that tells you whether your 555 blinks at 2 Hz, your filter rolls off where you wanted, or your level shifter actually shifts.

Benchmate runs real calculations only for a few shapes it recognizes: two-stage RC ladders, divider-biased NPN stages, and simple Boolean logic built from basic 74HC gate packages. Even then, those calculations use ideal models. For anything else, a clean summary means "structurally clean", not "working". The summary labels that difference. It says "Connection/layout checks passed" rather than "the circuit works".

2. Timing and transient behaviour

Benchmate does not run a transient simulation of its own draft before handing it to you. Start-up behaviour, oscillation, ringing, settling time, propagation delay, setup and hold margins: none of these are checked at the drafting step. The logic check uses ideal levels with no delay. The RC check is a frequency-domain calculation, not a time-domain run.

The app can simulate these things. Benchmate's checks just do not do it for you automatically.

3. Real part ratings and tolerances

A 1 kΩ resistor in a draft is an ideal 1 kΩ. Benchmate does not check:

• power dissipation against the resistor's rating;
• capacitor voltage ratings;
• transistor current or safe operating area;
• op-amp output swing for a particular part number (a few known parts have range checks, most do not);
• how the circuit behaves across component tolerance or temperature.

There are separate tolerance and corner tools in the app, but they are something you run, not something Benchmate runs on every draft.

4. Footprints and physical fit

The schematic carries package names, and the PCB tab turns them into footprints. Benchmate does not check that the package it picked is the one you can buy, or that a module's header matches the module on your desk. It also does not check that the board fits your enclosure, or that a part is in stock. The PCB exports pass the app's design rule check. We have not yet built and tested a board from them ourselves, so review them in your fab's viewer before ordering.

5. Firmware

Ohmeris runs firmware on the circuit for the supported microcontrollers: AVR (ATmega328P), PIC16 (PIC16F887) and a supported RP2040 path. On Pro and Premier, Benchmate can draft or edit firmware. A HEX file loaded on the virtual bench runs against your simulated circuit. Review the code and check pin assignments against the schematic. A successful simulation covers the inputs you exercised, so test the firmware and the circuit on real hardware before relying on them.

Why it says so instead of guessing

A language model is happy to write "This circuit will output a 1 kHz square wave at 50% duty." Sometimes that sentence is right, and sometimes the values in the drawing say otherwise. Early in development, we saw explanations describe parts and values that were not in the circuit the model actually returned.

So the rules are:

• When a calculation exists for the circuit's shape, its numbers come from the saved circuit, not from the model's text.
• When there is no calculation, the summary says the function was not verified.
• Findings the checks could not resolve stay visible after the last repair. They are not dropped to make the answer look finished.
• Edits to an existing circuit list what changed, what was kept and what was not checked, and Undo puts the old circuit back.

We would rather you get a draft with an honest "not checked" list than a confident paragraph you have to discover is wrong.

How to check it yourself on the bench

The bench exists for exactly this. A practical routine:

1. Read the findings first. Open the ERC panel and the chat summary. Fix or knowingly accept every open warning.
2. Check the obvious numbers. Use a multimeter on the supply rails and bias points, and the operating point for analog stages.
3. Probe the signal path. Put a scope channel on each stage. The four-channel scope lets you watch a signal change from input to output in one view. Our DDS synth build log does this with a PWM signal turning into a sine.
4. Decode the buses. For SPI, I²C or UART, put the logic analyser on the lines and read the decoded bytes. Check them against what your firmware should send.
5. Sweep it. Run an AC sweep for filters, or a DC sweep for thresholds.
6. Then treat it as a model. Everything above is software simulation with labelled models (ideal sources, idealised op-amps, modeled devices). It is a strong first check. It is not a measurement of a real board.

What we did not test

• We have not measured how often Benchmate drafts that pass every structural check also work as requested. Our internal runs show some do not, which is why the line above exists.
• We have not compared simulated results from any demo circuit against a physical build.
• We have not tested whether users follow the bench routine above, or how long it takes.