Skip to content

tests

Simulation testbenches. tests.kicad_sch instantiates them and is included from the root. The Key parts and Ports columns are generated by docs/gen_hardware_docs.py; edit only the Function text.

Sheet Function Key parts Ports
tests.kicad_sch Aggregates the benches below - -
differential_ring_oscillator_test.kicad_sch Ring oscillator start-up and steady state VPULSE -
oscillator_output_conditioning_test.kicad_sch Conditioning and /4 divider VPULSE, VSIN x2 -
long_tailed_pair_bias_test.kicad_sch Long-tailed pair bias point 2000 Ω, 3.3 V, VSIN x2 -
open_loop_oscillator_test.kicad_sch Small-signal open-loop gain at the ring-oscillator bias 3.3 V, 400 Ω, VSIN x2 -
BSS138AKS-QX.kicad_sch BSS138 I-V characterisation 1 mA, 3.3 V, BSS138AKS-QX -
power_supply_test.kicad_sch Fake USB/battery through BAT54C OR into 3.3 V booster (1.2 ms contract) BAT54C,215, PWL x3 -
integration_test.kicad_sch Full chain from battery discharge through USB takeover (130 µs contract) 1, 10k, 10p, BAT54C,215, PULSE, PWL x3 -

Enable a bench with exclude_from_sim and export from ../openorti.kicad_sch. Several benches use vendor models that ngspice rejects; see ../models/. power_supply_test uses the ngspice port ../models/TPS63001DRCR/TPS63001_TRANS_ngspice.lib.

Contract benches

tests.kicad_sch is the test registry: one bench per subsheet of that sheet. The runner reads the Sheetfile entries from tests.kicad_sch and does not glob the directory, so adding a schematic file alone does not make it a CI test. Files that are not registered are reported as SKIP.

A bench is CI-ready when its top-level text annotation contains @contract, ngspice .meas lines and @expect assertions. The runner exports the root schematic with only that bench enabled, runs ngspice, and checks every result against the contract. Benches without @contract are listed as SKIP, so the migration to contracts stays visible without blocking CI.

The text annotation is ordinary SPICE plus comments, so the .meas directives are also exported by KiCad:

* @contract
* @timeout 120

.tran 0.2us 1.2ms
.options method=GEAR RELTOL=0.01 TRTOL=7 chgtol=1e-13 abstol=1n
.save v(/tests/my_bench/Vout)
.meas tran vout_min MIN v(/tests/my_bench/Vout) FROM=0.1m TO=1.2m
.meas tran vout_max MAX v(/tests/my_bench/Vout) FROM=0.1m TO=1.2m
* @expect vout_min in 3.135 3.465
* @expect vout_max in 3.135 3.465

Net names are the hierarchical names produced by the root export, for example /tests/power_supply_test/3V3. The runner rejects duplicate reference designators and unresolved ${VAR} project variables, which catches subsheet exports that cannot be simulated reliably.

Supported assertions:

Form Meaning
@expect NAME in LOW HIGH LOW <= value <= HIGH
@expect NAME >= VALUE value at least VALUE
@expect NAME <= VALUE value at most VALUE
@expect NAME > VALUE value greater than VALUE
@expect NAME < VALUE value less than VALUE
@expect NAME ~= VALUE TOL abs(value - VALUE) <= TOL

@timeout SECONDS is optional. Keep it slightly above the expected runtime.

Running the benches locally

python3 hardware/tests/run_tests.py --list
python3 hardware/tests/run_tests.py --jobs 4
python3 hardware/tests/run_tests.py power_supply_test --keep

The runner needs kicad-cli (native, $KICAD_CLI, or the org.kicad.KiCad Flatpak) and ngspice (native or $NGSPICE). Each bench runs in a private temporary copy of hardware/, so the original schematics are never modified.

Running in the KiCad simulator

hardware/openorti.wbk is loaded automatically and keeps the KiCad simulator settings that the benches need:

  • Save all voltages is enabled, so any saved node can be plotted.
  • Save all currents is disabled. integration_test uses XSPICE behavioral models, and ngspice rejects .probe alli when the circuit contains digital nodes. Create a new analysis tab only after disabling Save all currents.
  • The workbook does not pin an analysis command. On the first run, KiCad adopts the .tran directive from the active bench automatically.

The project uses ngspice compatibility mode, the same as the CI runner. This avoids the PSpice/LTspice compatibility layer changing the behavior of the vendor and behavioral models.

Reports are written to hardware/tests/reports/:

File Contents
summary.md pass/fail table, failed requirements, skipped drafts
<bench>.md checks, actual values, broken requirements, log excerpt
<bench>.log full ngspice output
<bench>.cir normalized netlist used for the run
<bench>.raw.cir netlist exactly as exported by KiCad
<bench>.nets.txt net-name aliases, when normalization was needed
summary.json machine-readable result
junit.xml JUnit report for CI consumers

CI behaviour

The hardware simulation workflow runs on pull requests to main and on pushes to main. It does not hardcode a test list: the runner discovers the benches from tests.kicad_sch. The workflow first checks whether a simulation-relevant file changed:

  • KiCad sources: *.kicad_sch, *.kicad_pro, *.kicad_sym
  • SPICE decks and models: *.lib, *.cir, *.prm, *.mod
  • the runner and the workflow itself

Markdown-only changes skip the heavy job. The job always reports a status; when there is nothing to do it exits successfully with a short message. The CI workflow runs contract benches in parallel with --jobs 4, matching the 4-vCPU personal_vps. Raise it to 8 after moving to an 8-vCPU runner. Locally, --jobs defaults to the CPU count capped at 4. Reports are uploaded as the hardware-simulation-reports artifact and the summary is added to the job summary.

The job runs inside the official kicad/kicad:10.0.7 container. That image contains KiCad 10.0.7, ngspice 44.2, Python 3.13 and git, so the VPS host does not need KiCad or ngspice for CI. Host installs are still useful for running the benches manually.

Timing guidelines

Simulation time is dominated by the transient length, not by the number of checks. To keep CI fast without losing coverage:

  • Compress the fake-supply and load event schedule, but leave each phase enough time to settle. For the power supply, 100-200 us per phase is enough.
  • End the transient after the last behaviour worth checking; do not simulate a long stable tail.
  • Measure only the windows where the behaviour changes. Use one measure per requirement.
  • Use .save for the nodes you measure. Saving every internal node wastes memory.
  • Prefer .op, .dc and .ac for bias, I-V and small-signal benches.
  • Keep the maximum timestep at the smallest feature you need (for example 0.1-0.2 us for the 1.5 MHz booster). A smaller timestep does not improve the contract and costs CPU time.
  • Split a long bench into independent shorter benches when wall-clock time matters; the runner can execute them in parallel.

Current contracted benches:

Bench Typical time
power_supply_test about 25-30 s
integration_test about 150 s
oscillator_output_conditioning_test about 5 s
differential_ring_oscillator_test about 3 s
long_tailed_pair_bias_test under 1 s
open_loop_oscillator_test under 1 s
BSS138AKS-QX under 1 s

The integration test closes the loop with the behavioral 4040 counter. The vendor 4040 transient model is too slow for CI; keep it for detailed offline simulation.

integration_test exercises the actual power supply and ring oscillator, then closes the chain through test-specific copies of the production output conditioning and measurement sheets. The fake supplies discharge the battery while USB is unplugged, then ramp USB in at the depleted battery point so the handover is part of the contract. Those copies keep the actual LVDS receiver and the real hierarchy and wiring, but use behavioral LVC1G80 and 4040 models. The behavioral 4040 VCC is fed from the booster through a 1 Ω series resistor (RMEASVCC), so the whole chain shares the single 3.3 V rail. The real LVC1G80 divider is still covered by oscillator_output_conditioning_test. This keeps the full-chain transient around 150 s instead of tens of minutes. The helper sheets live in tests/integration/ and are not separate benches.

Runner requirements

Resource Recommendation
CPU 4 vCPU on the current VPS (--jobs 4); 8 vCPU for --jobs 8
RAM 8 GB ample; 16 GB headroom for larger models (one simulation uses about 30 MB)
Disk about 4 GB free for the kicad/kicad:10.0.7 image plus reports
Software CI image: KiCad 10.0.7, ngspice 44.2, Python 3.13; no display or GPU

Each ngspice run is single-threaded, so CI speed scales with cores until memory or the number of benches becomes the limit. Pin the KiCad and ngspice versions on the runner; tolerances should be wide enough that a simulator upgrade does not turn a real design change into a flaky test.

Migrating a draft bench

  1. Add a top-level text annotation with @contract, .meas and @expect.
  2. Use the hierarchical net names from the root export.
  3. Add .save for the measured nodes and a @timeout.
  4. Compress the event schedule and measure only the useful windows.
  5. Run python3 hardware/tests/run_tests.py <bench> until it passes.
  6. Update the Function text in the table above and commit the bench.

If the bench is not already a sheet in tests.kicad_sch, add it there as a subsheet of the test sheet. The runner only executes registered benches.